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ABSTRACT 

Changes that are occurring in work and family life have implications for 
vocational education. The usefulness of job and task analysis is becoming more limited 
as rapid changes in workplace and family activities reduce the predictalnlity of taskii 
and activities, and as mental processes that are not directly observable become more 
central in these tasks and activities. This study sought alternative af^[>roaches to 
vocational education curricular and instructional development by applying cognitive 
theory to the development of an instructional design model Four factors were central 
in the instructional design model: (a) fidelity of the learning situation to die real 
world, (b) visualization, (c) range and depth in learners' experiences, and (d) mediation 
of learners' interpretations of their experiences. A computer-based, level m video-disc 
learning environment was designed as an instructional prototype based on ttiis model. 
The intermediate level of the prototype was develc^wd and tested in this project 
Learning gains were found on all assessments used Findings supported the 
effectiveness of the intermediate level learning environment in developing learners' 
knowledge and thinking relevant io work activities. 
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CHAPTER 1 



IMTRODUOTION: PROBLEM. RATIONALE. AND PURPOSE 

The itseaith reported in Ais document was completed as part of ttie Higher Order Thinking 
Research Program in the Minnesota Research and Devdo^ The purpom of this progrwn 

of leseaich aie (a) inoeased understanding of tfie role of vocational education in facilitating higher 
Older flunking capacities in learners, (b) increased understanding of Ae relationship of order 
Ainking processes to th« roles vocational education prepaics indhdduals to assume, (c) ways of 
teaching Aat support leamere in developing higher order thinking capacities, and (d) ways of assessing 
learning of thinking processes. This program of research was initiated in 1986 witti the development of 
an agenda for inquiry regarding vocational education and higher order dunking (Thomas & Litowitz, 
1986). Purpose (b) was addressed by ttwee studies Aat examined ttie nature of thought and knowledge 
underlying competence in work and/or family roles (AnderKm, 1989; Cooke, 1988; Johnson, 1987, 1988; 
Thomas, 1988b). Purpose (d) was reflected in a project whidi developed and tested a new approach to 
assessing Ainking wiAin Ae context of work sihiations (Thomas, 1968a). Purposes (a) and (c) are 
addressed by Ae lesearch leported in Ais pubUcation and its companion Volume I (Tnomar ft 
Englund, 1989). The assessment material Aat was developed and tested in 1988 is incocporated in Ac 
research reported here. Space is not devoted here to discussion of Ae meanings of hi^er order 
thinking since Aese are discussed in Ae companion Volume L 

Vocational education has historically prepared people for woric and family life. Vocational 
preparation has traditionally helped learners acquire understandings and skills whid\ enable Aem to 
engage in activities relevant to an occupation or family life. Job and task analysis has been a central 
approach to vocational education curriculum development and instructional design. This approach 
involves observing people engaged in Ae activities of interest, breaking Ae activities down into 
component operations, and identifying and sequencing experiences Aat %vill enable learners to master 
Ae operations and assemble Aem to perform Ae activities. Improvements in vocational education 
curricula and instruction have focused on improving ways of doing job and task analysis, as well as 
expanding its Kopt, bettering testing strategies and procedures, and improving Ae effectiveness of 
learning experiences in producing acquisition of understandings and skills. 
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Change in the Nature of Work 
The job and task analysis approach has been functional as long as work and family life activities 
have remained relatively stable, and as long as activities of interest have been able to be physically 
observed. Rapid changes occurring in ttie workplace and in ttie family are reducing staUlity and 
introducing key activities that are not physically observable. Tedrndogy has been a m^or factw in the 
rapid changes which have reduced the stalrility and predictability of ttie nature of work activities. It is 
no longer reasonable to assume ttiat work activities considered to be central to a job or occupation at a 
given time %vill continue to be as important five years later. It may not even be reasonaUe to assume 
that jobs existing today will exist in five years. Thtis, focusing curriculum and instruction solely on 
existing activities is becoming a questionable practice. In addition, work and family activities emerging 
during these dtanges depend more heavily on complex mental processes that carmot be directly 
observed. Technology, which now performs many physical operations that used to be done by people, 
requires skilled operators and maintenance by peofde who have sophisticated knowledge. Tasks 
central in operating and maintaining this technology are likely to be mental rather ttian physical. Thus, 
those tasks which can he physically observed are not the central tasks being performed. In this way, 
technology has increased the predominance of mental operations relative to physical (^>erations in 
work activities. 

There are numerous examples of shifts toward greater emphasis on mental activity in work roles. 
Drafting is no longer as dependent on physical manipulation of drafting tooU; computers generate tfie 
output of drafting processes. As a resuJt, the importance of physically manipulating drafting tocds has 
declined and drafting has become more exclusively a matter of mental operations. Observing someone 
physically operating a computer does not provide an understanding of ttie mental activities central in 
the task of drafting. Computer-driven machining, which has depended on knowledge of metals and 
machines, and skill in the physical manipulation of machines, now depends heavily on the mental task 
of programming computers to drive the machines. As agricultural markets become increasingly 
worid-wide in scope, agribusiness management activities are becoming more complex. Frequent 
decisions about lines and stocks of products, as well as which enterprises to continue and which to 
modify or abandon, hav^' become necessary. These decisions invdve consideration of a wide range of 
factors which interact to produce a higli degree of uncertainty. Computerization of offices has brought 
with it the need to understand and manipulate complex software programs and to effectively structure 
management information systems for highly specific purposes. Such activities draw on an array of 
complex mental capacities. 

Citing such examples carries ^he risk of im|:dying that physical operations don't involve 
knowledge, mental capacities, and operations. This is not the case. The point is that complex activities, 
such as the examples cited above, are largely mental in nature-even when the desired object and some 
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aspect! of die activity are phyaical (NeweU k Simon, 1972^ The mental aapccti of the activities are 
central to tfie outcome or mult of Ae activity, and ttiey cannot be obaenred in Ae same way as 
physical aC:>'/ities. 

Functions and tasks for which technology does not exist are often characterized by a high degree 
of variability and complexity. Examples are functions and tasks that require inteqpretation of social 
signals or moie extensive understanding and interpretation of human beings (who vaiy widdy and are 
extremely complex). Such functions and tasks are largely mental in nature. 

Changes in Fandhf life 

The family has also been touched by technological and social dumges. Ways of linking about 
family roles have become more flexible as multiple members have taken on responsibilities for income 
production, househdd management, and child care. As Aey have become more varied and complex, 
logistics, patterns, and coordination of everyday activities of family members invoh^e increasingly 
complex mental woric The simultaneous involvement of several members of a family in work and 
educaticmal pursuits has necessitated selection and use of services outside Ae family to meet family 
membere' day-to-day needs. Goods and sendees that used to be physically produced by families arc 
now selected through decision processes that involve cojiplex mental operations. The wider and more 
varied involvements of family members have made family rdes, relationships, and meaiungs of family 
more complex, as well as more ambiguous and open to revision. Family roles and activities are likely to 
be determined less by adoption of a past or existing pattern Aan by decisions and dioices that entai! 
complex mental processes. 

Alternative Stmtepes far Dealing with Change 

As work and famUy activities become less predictable, and as complexity heightens Ae importance 
of intellectual processes not ea^ly observed, vocational educators face Ae need to eiAer modify or 
rethink Aeir approach to curriculum and instructional develqfmient As concerns about attitudinal and 
motivational issues, eAics, human rights, cultural diversity, and oAer affective dimenrions of work and 
family life increase, vocational education is challenged to go beyond its traditional interest in 
performance to address questions of meaning. 

One approach vocational educators migfht take is to speed up curriculum and instructional 
development processes. As new work activities develop, Aey could be observed and analyzed, and 
curriculum quickly devised to prepare peqfrie for Ae activities in a shorter amount of time. This mig^t 
be viewed as a 'rapid-response* if>proach. At least Aree factors mitigate against Ais solution. One is 
the expense of task-analysis^ented curriculum development processes. It is time consuming, and 
additional resources to support more frequent activity of Ais type are unlikdy to be available. Second, 
this approadi does not respond to Ae need to understand mental activity as an increasingly central 
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factor in work roles. Third, this approach docs not address personal meanings and implications of 
change, issues that are closely related to people's attitudes and motivations. 

A second approach involves preparation of carefully prepared instructions, often in the form of 
detailed operations or procedures manuals. Thb approach may be feasible in situations where 
operation or procedures are not too complex, but fails as tfie complexity of operations and procedures 
increases. Anyone lacking a l>asic understanding of computers or familiarity wiHi the structure of a 
computer program, and who has tried to interpn^ computer manuals for complex programs, has 
experienced the problems with this af^>roach. 

A third approach vocational educators might take addresses the need to focus attention on mental 
capacities and operations. This approach, known as cognitive task analysis (Gagne, 1977; Landa, 1974; 
Lesgdd, Bonar, et al., 1986; Lesgold, Lajoie, et al., 1986), defines cognition in terms of mental tasks, and 
uses the task analysis process to create hierarchies of nested tasks invdved in an activity. These 
hierarchies are created by using either rational or empirical approaches to task analyus, or some 
combination of the two. Mental operations are decomposed into their component parts. The parts are 
dien taught and assembled during the learning process. A similar approach has been used in the 
affective domain to analyze affective understandings and skills (Posner & Rudnitsl^, 1986). While tius 
approach succeeds in shifting the focus to mental aspects of activities, and is beginning to focus on 
attitudes and motivations, its basis in already-defined, existing activities resembles job and task 
analysis. This approach appears to be effective and most usat>le when activities and proUems are 
well-defined; that is, wh^ goals are given and mental procedures and processes for accomplishing 
them can be deariy specified in detail. Thus, this approach hdds promise for curricular devdopment 
and instructional design, when the desired outcome is learning mental or affective aspects of existing, 
well-defined activities. Learning hierarchies may be useful in combination with the rapid-response 
approach described previously. They are less useful for activities in which goals and procedures cannot 
be completely determinefj in advance. Such activities include those that do not yet exist and those 
which are "ill-defined." Hi-defined activities are those in which goals and procedures must be 
constructed. While the affective version of cognitive task analysis addresses attitudes and motivations 
on a performance level, it does not address these ireas at a meaning level. Consequentiy, its impact on 
tiiese areas may be superficial rather than deep. Further, like job and task analysis, cognitive task 
analysis is time consuming and expensive. 

Purpose and Objectives of the Research 
The emerging importance of intellectual processes in changing woric and family activities points 
to cognitive theory and research as a promising source of theories and concepts on which to base new 
approaches to curricular development and instructional design. The cognitive-task-analysis approach 
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itflects a move in this direction. The purpose of the research reported here was to apply previously 
identified cognitive ttieory and research (Th 3mas Utowitx, 19tt) to tte development of an 
instructional design model for vocational education that would reduce the limi^tions of the approadies 
identified above, as well m more adequatriy address ttie range of changing conditions in the woricplace 
and Ae famUy. More spedficaUy, ttie intent was to develop an instnictionrf design modd that would 

(a) reflect the interests of vocational education in real-world work and famUy contexts and acthrities; 

(b) focus attenticm on the mental aspects underiying work or family acthrities; (c) address learning goals 
Aat indude performance, but ttiat also extend to underiying meanings; (d) reduce Ae separation of 
cognition from affective and psychomotor aspects of human acthdties; (e) prepare learners for tt\e world 
as it currentiy exists as well as ttie yet-to-be-determined, nol-yet-existing world of Ae future; and (0 be 
reasonably practical for vocational education to tmfriement in terms of effort and cost Specific 
objectives of the research reported here were to: 

1. Develop an instructional design model for teadung mental aspects underlying work and family 
activities in a context of rapid change and hig^ tedmdogy. 

2. Test the instructicmal design model by developing a prototype instructional design for teaching 
mental aspects underlying activities widtin a domain relevant to vocaticmal education. 

3. Develop materials required by the prototype instructional design. 

4. Test the prototype instructional design for effectweness in facilitating development of tt\e 
mental aspects underlying activities within tlie sdected domain. 

In addition to die importance of providing a test of die instructional design model, an 
instructional prototype was also seen as providing a concrete example of tt\e iq>plication to ir^^Jruction 
of concepts antidpated to be new to vocational educators. 
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CHAPTER 2 



UNDERLYING THEORY AND RESEARCH 

A considerable body of research exists tfiat focuses on cognition in real-worid contexts. This 
leseardi has investigated tiie tiiinking underlying activities of people in a wide range of activities 
which include playing baseball, technical trouUe^ootingr bookkeeping, invertment decisions, making 
dairy delivenes, and nonformal teadiing (Bhaskar, DiUard, & Stephens, 1983; Giiesi, Spilid), & Voss, 
1979; Qarkson, 1962; Luria, 1976; Rogoff & Lave, 1984; Ramsussen k Jensen, 1974; Soibner, 1984). 
MuA of this research has compared people who have readied a hig^ levd of expertise in a role, 
function, or activity, wth those who are less competent or who are learning an acthdty. This research 
has revealed a dose relationship between people's tiiinking processes, ttie nature of their knowledge, 
the meanings they ascribe to events or phenomena, and their levd of competence. For example, one 
study completed by vocational education researchers revealed relationships between tedtnidans' 
cognitive flexibility and their knowledge structures (Smitti & Moss, 1970), This body of researdi has 
revealed (a) tiie importance of domain knowledge and context in learning and dunking; (b) that, in 
addition to content differences, knowledge domains also differ in tiieir organization; and (c) tiut it is 
possible to identify levels of progression or development of people witiiin a knowledge domain by ttie 
type of content people express and its organization. 

Expert-novice comparison studies com|:deted as part of tiie Hi^er Order Thinking Research 
Program in the Minnesota Research and Development Center for Vocational Education (Anderson, 
1989; Qx*e, 1988; Johnson, 1987; Thomas, 1988b) have indicated ttiat individuals vnth more extensively 
developed knowledge structures attend to deeper, prindple-related features of situations and use more 
powerful, inclusive concepts and jnindples in ttidr interpretations. In contrast, less-expert individuals 
focus on surface features detectable through ttie senses and are more cone ?tc in ttieir interpretaticMis. 
This reseaidi indicates that complex judgment and problem solving involve strategic, selective focusing 
of attention on cues ttiat aie relevant to a proUem or goal. Knowledge struchires guide ttiis selective 
attending and are key elements in the interpretations and meanings people generate, as well as in 
people's effectiveness in dealing with problems and goals. Cognitive research suggests ttiat knowledge 
structures which support complex thinking are organized in a way that intcgnHet (a) features of the 
external worid, (b) goals or intentions, (c) actions, and (d) domain-knowledge concepts and prindfries. 
The organization and content of such knowledge structures is quite different from ttie disdplinary 
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structures of knowledge reflected so often in printed curricula. While disciplinary concepts are present, 
they are embedded within action* or meaning-oriented structures that guide interpretations and 

actions. 

The idea that knowledge domains differ, and that ttiere is a dose, inseparable relationship 
between knowledge and thinking, is reflected in the cognitive ttieories underiying the research 
reported in this publication. The other central feature in ttie sdection of theories on which tius wcMk 
is based is the theoi/s ability to integrate internal states of people, features in the external worid, and 
people's acticms. Internal states of human l)eings of particular interest indude people's goals, 
intentions, meanings, and their domain-knowledge concept s and prindples. The follo'ying sections 
outline the theories that form the basis for ttie instructional design model reported in Chapter 3. 

Task Emrironments 

The concept of a task environment is central in Newell and Simon's (1972) theoty of human 
problem solving. This theoiy distinguishes between an environment (which is external to a problem 
solves) and the problem solver's internal representation of the environment, and focuses on two 
interacting aspects: (a) the demands of task environments, and (b) tfie psychological state of actcm in 
terms of intentions and what they know about a task environment ttiat is relevant to those intentions. 

The External Environment 

Because a person's psychological state affects how an environment is perceived, a person's 
description of an environment reflects their interpretation of the environment in terms of tiie goal, 
problem, or task they have in mind, as well as in terms of the strife of their knowledge. EnviixMiments 
that can be readily described by almost anyone in simple ter^is can be ea«Iy communicated Those for 
which this cannot be done (those which are described very differently by different pec^le) cannot be 
communicated in a way that assures a consistent message will be received. Complex environments m 
present in this latter group. For example, a description of a four-legged, 36-inch-square, solid walnut 
table is more likely to be received dS a similar message by many individuals, than is a description of a 
car's electrical system or a complex human relationship. 

Internal Representations of the External Environment 

A task environment is an environment coupled with a goal, problem, or task (Newell & Simon, 
1972). A task environment includes features, people, objects, events, and relationships in the external 
worid, as well as a person's internal representation of those features, peofrfe, objects, events, and 
relationships. A person's internal representation of an envircmment is selective; it contains only those 
features, objects, and relationships that are relevant to the person's goals, intents, and interests and that 
the person's knowledge allows them to perceive. These internal representations guide a person's 
strategies in planning their moves within the task environment. The nature of a person's internal 
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rtfmsentation reflects the interaction of a person's psychologic^ state (ut., Ae person's goak or 
motivations and their concepts) with the actual external environment Numerous possiUe 
^presentations are possible for all but Ae most simple environments. The same external environment 
isivpraentedasdiamnttMkcnrm^^ and differing 

levdsof knoivledgc: The goal accepted by ttie person as somettiing to be attained in a given task 
environment is the connecting link between ttie external environment and ttie person's psychological 
state 

Coals, Intentkms, and Adkms 

Newell and Simon's (1972) theocy of human problem solving takes into account ttiat a person's 
knowledge and goals influence what they see and pay attention to in an environment Sdiank and 
Abelson (1977) proposed a dieoiy of knowledge structures that focuses on ttie nature of people's 
knowledge and fht relationships between ttiat knowledge and ttie meanings, goals, and actions a 
person formulates or sees in odiers. Like Newell and Simon's ttieory, Sdunk and Abdson's ttieofy 
assumes a close relationship between knowledge and thinking, and between content and process, and 
integrates conditions and featuies in ttie external worid witti psydiotogical states of people. The 
limitations of Newell and Simon's theory imposed by its focus on performance are reduced by 
combining it wi A Schank and Abelson's ttieoiy, whidi extends to learrting and development (Sdumk, 
1982). Schank and Abelson's knowledge-structures ttieory also extends beyond goals and task- 
environment knowledge to meanings as they are reflected in personal belief systems and ideologies 
which dispose people toward certain orientations in ttieir thinldng and acHons. 

Schank and Abelson's theoiy pertains to ttie real-world actions of people, and rdationships 
among actions, purpose, and context This ttieoiy of knowledge structures fofmses on human 
intenti(ms, dispositions, and relationships. In addition, the ttieoiy adoiowleJges the messine^s and 
frequently illogical character of die real wmld. This theoiy of scripts, plans, goak, and ttiemes holds 
ttiat peq>le understand and act in real-world events based on a knowledge sy^stem omsisting of 
varying types and levels. Central tenets in this theoiy are: 

1. Knowledge domain spedfkity. Knowledge in one domain may be oi^ganized according to 
principles ttiat differ from those which organize knowledge in anotfter domain. A ttieory of 
cognition in a particular domain requires a theoiy of ttie domain itself*-a description of ttie 
more complex structures in memory and their characteristic properties. 

2. Practical reMonin^ Procedures for applying past knowledge to new experiences often seem to 
require common sense and practical rules of ttium.b. If rules of thumb are not available, 
reascming about specific actions for specific ptotriems or situations in addition to, or instead of, 
formal analysis is used. 

3. Schenaiind knowledge. Knowledge is organized in chunks or packages so that with one or 
more cues, many inferences are possible about what might happen in a given sihiation. 
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4. Multi-levd knowledge atnidiiitt. Knowledge is oi^ganized in multi-levd structures which 
include (a) small units of meaning reflected in statements of description or rdationship at the 
lowest level; (b) structural links ttiat specify types of relevant information at tfie middle level; 
and (c) large, al)stract, over-ardung ttiematic structures at ttie top levd, which oiganize and 
index the more ^>edBc information contained in ttie lower levds. 

5. Functional flexibility. Knowledge structures have functiorkS*-ttiey aid comprehension^ organize 
memoiy, and guide learning and abstraction. Th^ are flexible in tftat ttie mental processes 
that use them can be combined to serve multiple purposes. 

The multi-level knowledge structures in Schank and AUeson's (1977) theoiy range from scripts at 
the most specific level to themes at ttie most general level Individuals determine meaning for events 
by placing a piece of information in context A plan is sensiUe if ttte goal it leads to is understood; a 
goal is understandable if it is part of a laiger theme. These types of knowledge structures and ttieir 
types of content and functions are described in the following i^aragraphs. 

Scripts 

Scripts contain appropriate sequences of events in a particular context (Galambos, 1986; Reiser, 
1986; Schank, 1982; Schank & Abelson, 1977). They reflect stereotyped sequences of actions flut define 
well-known situations. Since scripts are assembled to suppcMt routine actions which require minimal 
amounts of conscious processing, they are not a central focus in tfie research reported here. An 
example of a script familiar to many people is eating in a restaurant 

Plans 

An unfamiliar situation ^vhere no scripts apply requires ]:rianning. Planning is harder flian using 
a script because it requires a great deal of thought about conditi<ms of af^icability. Planning is ttte 
reasoning by which an individual decides upon one or more actions (Abbott & Blade, 1986; Leddo & 
Abelson, 1986; Schank, 1982; Schank & Abelson, 1977), the choosing of a path to a goal Plan 
knowledge structures are made up of general information alxnit how people achieve goals. A plan is a 
series of actions jne could or would perform to attain a particular goal. Plans are rule-bound 
knowledge structures that have the following funcHons: (a) describe the set of action choices a person 
has when setting out to accomplish a goal; (b) allow a person to detect, understand, and judge the 
relevance of particular methods to the achievement of particular goals; (c) allow actions tiiat did not 
work to be adjusted; (d) enable access to information about possible problems which could arise in 
activities; (e) enable explanation of sequences of actions in light of a goal they are intended to achieve; 
(0 enable predictions about what will occur in an event; (g) connect cues in external environments, 
goals internal to people, and acts that can be carried out in the external environments to achieve those 
goals. Far-reaching goals require more planning to achieve than do less far-reaching goals, because 
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they often mjuire that smaller subgoals be identified and tfut plana be developed to achieve tfioae 
subgoala. 

The above discussion of plans indicates ttiat both planning and understanding use plan 
knowledge structures. However, constructing plans and understanding plans are different processes. 
Constructing plans invdves choosing a path to achieving a goal, stringing methods together in an 
admissible or optimal way to adiieve a given goal Plan creation is problem solving. Understanding 
involves creating connections, not creating one's own plan. Understanding involves reconstructing 
someone else's plan, ascertaining a person's goal and recognizing tfieir actions as attempts to realize 
that goal By finding a plan in one's own knowledge structure, a person can make guesses about tiie 
intentions underlying another^s actions. Plan understanding invc^es veiy broad knowledge of large 
numbers of actions and goals, and also allows evaluation of one's own or anottiei^s plan as to whettier 
the plan was well formulated or is the best that could have been developed 

Goals 

vs implied above, plans are linked to goals. A goal is die item, event, or state desired and 
typically invokes a dunge from a current state to a desired one. Goals are intentions tftat arise from 
individuals' states and from the interaction of those states widi external environments. Goal structures 
have been diaracterized as hierarchical structures consisting of main goals and subgoals (Abbott k 
Black, 1986; Galambos, Abelson, & Black, 1966; NeweU & Simon, 197% Schank & Abelson, 1977). The 
highest level is the main goal. Main goals organize subgoals. Subgoals contribute to and actually 
accomplish main goals. Instrumental to the subgoals are plan actions (already described) which m>ear 
at lower levels in the goal structure. Goals aq^pear to have die veiy crucial function of structuring and 
indexing information in memory so it can be located when it is relevant (Anderson, Spiro, & Montague, 
1977; Bobrow & Collins, 1975; Glass, Holyoak, & Santa, 1979; Reiser, 1986). 

Themes 

Themes arc broad, overarching principles, and concepts that give rise to goals. Themes generate 
and reflect clusters of related goals that tend to occur togetiier. Themes make goals understandable in 
that they provide a laiger context for isolated goals (Kay & Black, 1986; Schank, 1982; Schank k 
Abelson, 1977; Seifcrt, Abelson, & McKoon, 1986). These general, high-level knowledge structures store 
together in memory a wide range of highly varied situations from a variety of contexts. Constructed 
from experiences with specific cases, events, or activities, themes organize episodes which may vary 
greatly in tiieir specific feahim into patterns that reflect abstract similarities. Because tiiemes are based 
on thematic similarity, they are context independent Thus, tiiey can facilitate transfer of goals and 
plan actions across a wide range of types of situaticms. Themes provide knowledge relevant to making 
predictions, deriving expectations and explanations, categori^ng, and planning. For example, themes 
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contain tfie background information ttiat enables predictions d an individual's goals. Themes concern 
types of goal relationships, opportunities, and difficulties often encountered in pursuing goals, tt^d 
implications for goal success or failure. Like ttie goal-action structure, ihemes are believed to be used 
both in organizing memory to aid retrieval and in interpreting new experiences. Types of tiiemes 
include role themes, interpersonal titemes, and life titemes. 

ImpliaUions of Task Emrinmments and KnowUdge Structum for Vocational EducaHon 
V/hat insists do the tiieories described above and tiie cognitive research involving people at 
varying levds of competence provide regarding tiie design of iiutruction? What questions do ttiey 
raise? What are tiieir implications for vocational education? This section will t>riefly address titese 
questions. 

Person>Environmcnt Interaction 

The theories reviewed in tiie previous sections suggest a dose rdatioruhip, or interaction, 
between people and their environments on several levels. On tiie psychological level, people's goals, 
intentions, and motivations influence what they perceive in a problem or situation. Also on tiie 
psychologic^ level, people's knowledge states influence what tiiey perceive. Peofrie at different levels 
of develo^nnent in task-environment-relevant knowledge will perceive essentially different task 
environments in the same external environment Fu ilh ermore, tiie same paths in task environments rjrt 
not available as paths to the goal (plans) for people at different levds of devdopment (Newdl & 
Simon, 1972). An implication for instruction is that instructors cannot assume that environments to 
which learners are exposed during instruction are tiie same for all learners, or are tiie same for learners 
as those environments are for an instructor. Learners will experience environments in tiie way tiieir 
level of knowledge and thdr god state orients tiiem, not necessarily in tiie way tiieir instructors 
intend. An understanding of tadc environments, tiieir demands, and tiieir meanings for people witti 
different gods and levels of knowledge, requires study of extemd environments ttirough tiie eyes of 
people at varying levels of experience and knov^ledge. On a phydcd or sodd level, people's acts in 
environments affect environments in same way. Physicd or sodd changes in environments also have 
an impact on people's knowledge. Thus, to think of "the* task^ or *tiie" environment, is to oversimplify. 
Instead, a concept of learner environment interaction is suggested. 

Spedfic Nature of Domain Knowledge and Task Environments 

At first glance, knowledge-domdn spedfidty seems troublesome, espeddly given tiie issues raised 
in Chapter 1 concerning rapidity of change and tiie need for flexibility in applying knowledge to new, 
unfamiliar situations and problems. If knowledge needs to he specific, how can it be generalised? It is 
important here to distinguish between specific domdn and spedfic knowledge. These tiieories suggest 
the importance of a deep understanding of the nature of the knowledge domdn of interest, induding 
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it» higher Icvds, the nature of the ttiinking processes Aat drtw on it and its central task environments 
conceptuaUzed at vaiying levels of abstraction. BoA more general concept- and prindple-level 
knowledge and mow specific, factual levd (spedlk) knoi^^ Further 
clarification of this point will be made in subsequent sectkms of this publication. 

The Need for a Picfurg nf Dcvdnprn^t Within a Domain 

The importance of a person's knowledge state in terms of what ttie peison percehrcs and attends 
to in an envifonment suggests Aree tilings (a) that it is important to know ttie extent of prior 
knowledge development of learners before assuming the appropriateness of any ghrcn instnictkm; (b) 
Aat because people at different leveU of knowledge do percdve and focus on very different thfaigs in 
an environment their perceptions, intcrpretatkms and analyses o«er vahuMe chies to ttieir stage of 
knowledge devek>pment in a domain; and (c) that the characterization of devek)pment needs to be 
specific to the domain in order to be highly useful for instnictkm. 

The Kind of Knowledge that Should Be Taught 

A deep understanding of the knowledge domain is likdy to assist in determining where to focus 
instruction. An understanding of the knowledge domain wiU darify tfie function of the different 
structuits it contains. Plans generate actions; goals generrte plani^ themes generate goals. Small-scale 
social and technological change is likely to bring about the need for changes in plans. Such changes 
lequife alterations in the ways people do things and how certain goals are achieved. Laiger-scale 
change brings about changes in goals. Paradigmatic change is theme-levd dumge. An underrtanding 
of change in terms of the knowledge levels for which it has impUcations is an important basis for 
decisions about the aims of curriculum and instructujn. 

Ways of Teachi ng the Kind of Knowledge that Should Be Taught 

While the theories discussed here are usehd in aiding underrtanding of rdationships between 
people's mental states and their interpretations and actions in environmenti, tfie theories are of more 
limited help in iUuminating how people develop task-environment representations, plan*, goals, and 
themes, and how such development mi^t be facilitated. Some reflection on the theories and Sdiank's 
(1982) more recent woric gives some dues, but beyond these, oAer theories are needed to address the 
question of leammg and teaching. Such theories will be discussed in the next main section, entitled 
Research on Development and Learning. 

The theories of task environments and knowledge structures do suggest some instructional 
(mictices. These practices indude: 

1. Being thou^tful aiXHit the levd at which task environments of interest in instruction are 
conceptualized and the levd at which instructional goals are set, and hdping learners develop 



13 



conceptualizaticms of relevant task environments based on deeper features that are linked to 
principles. 

1 Enabling learners to develop goal-actkm structures by involving learners in identi^g goals 
underlying situations and actions, in searching for and predicting actioni that relate to ^vcn 
goals, in «plaining why certain actions were done, in critiquing actions witfi re^^ect to tfieir 
consequences for goals, and in underManding odicn" explaiuitions^ prediction^ and critiques 

1 Enabling learners to develop tfieme structures by eqKMing them to many cases that differ in 
context but are similar in tfiematic principles^ aiui by engaging leanters in explaiiUng goals in 
terms of themes, in considering die conaequencet for tfiemes of holding alternative gMls, in 
predicting goals given themes, and in critiquing goals in terms of tfiemes. 

4. Hel{Hng learners to develop die habit of, and capacity for, doing tfiese things on dieir own, 
independent of the teacher. 

Examples of Applications in Vocational Educarion 

The interest reflected in the tiieories described alxive in real-world environments ^d events, and 
in the organization of knowledge for understanding real-worid actions of people, is relevant to 
vocational education. Domain-specific knowledge structures and thinking processes studied over tiie 
past five years in the context of vocational education in research sponsored by tiie Minnesota Reseaith 
and Development Center for Vocational Education, and in doctoral dissertations completed in tiie 
Department of Vocational and Technical Education at tiie Umversi^ of Minnesota, reflect tiiese 
theories. One study explored knowledge structures and tiiinking processes underlying techiucal 
trouUe-shooting Johnson, 1987; Jc^nson, 1988). In these task environments, tiie main ^oal is usually 
given (e.g., to find the fault in a device or system) and the person^s task is to generate subgoals and 
select or develop plans that will accomfrfish the subgoals. Finding tiie fault in a device or system is 
facilitated by an understanding of the subsystems existing in that device or system, and how they 
function and interrelate. The knowledge structure underiying tius kind of activity as eq^erts perform it 
is a mental model of the task environmer.t (the system and its subsy^^tems) (Rasmussen & Jensen, 1974). 
Thinking is driven by hypotheses generated about tiie states of the subsystems. These mental, 
model-type structures are relevant to industrial education and agricultural mechanics, and, with 
extension to biological systems and subsystems, to health education and to plant and animal health in 
agriculture. Mental models may also be potentially relevant in business, marketing, industry, and home 
economics areas that focus on production and distribution systems. 

A second type of knowledge structure was revealed in studies of knowledge domains underiying 
interactions between peofde. These structures were found to be oiganized in terms of contextual, 
ph)rsical, and social features of task environments and on states, conditions, goals, and meanings of 
people (Cooke, 1988; Thomas, 1988a). In such task environments, main goals may be given. Often, 
however, main goals as well as subgoals and the plans for achieving them must be established as the 



14 

25 



interaction proceeds. Thinking procettcs focui on interpretation of itatei, on invoking rdevint themes, 
and on goii and plan sder^on and formuution. Theie atructurea are relevant to vocattonal education 
in home economics aieas including pafen ting, chikl care, f aauly life eduratkm, as weU as to the sales 
aq>ect of maiketing education, and ttie preparatkm of supeiviaot^ tninen» and teadmt. 

Athiidtyi^of domain-spedfic knowledge stnicture studied is that undcriying complex de^ 
ordioices. The je leqdte finding a sufficient or optisaal BMtdi between a 

amy of alternative options or strategies. This kind of knowledge structure was studied in the context 
of housing decisions (Anderson, 1989^ Earfier studies expkxing this type of knowledge structure in the 
context of ecofiomic investment decisions (Oaikson, 1%2) 

whidi conditions and options were filtered to atiive at a SMtdu Examples of vocational education 
areas in whidi tius type of knowledge structure has potential rdevance indude economic management 
and decision making, and strategic planning rdated to business and housdtcrid management amf 
enterprise management in agriculture. 

It is antidpated that there are a number of additional types of domain-spedfic knowledge 
structures ttiat are also rdevant to vocational education* Furthermore, ttie types of knowledge 
structures identified above may be used in combiiution witit eadi otiier in certain kinds of task 
environments. These issues need darification tiuoug^ additional researdt The main points to be 
made here are tiiat (a) die organizaticm of knowledge appeare to differ in rdation to the kind of task 
environment being dealt witti, and (b) ttie Wnd of dunking ttiat is done rrgarding a task environment 
appears to be assodated with the type of knowledge organization invoked. Differences in knowledge 
structures leflect differences in the nature and structure of task enviroiunents, botii in tiie degree to 
which tasks require that goals must be generated, and in die ruture and variability of ttie goals. The 
knowledge^omain studies leferred to above support Newell and Simon's (1972) point tiut tiiere is a 
crudal inlwdependence between task environments and goals. This interdependence allows 
task-relevant components of the task environment to be separated from task-irrdevant components, and 
enables knowledge about task environments, i^ans, and goal-{^ relationships tiiat peqde have 
available to be brought to bear on tasks or problems. 

Resmdt on Deodopment and Learning 
Cognitive research suggests that individuals who have reached advanced levels of understanding 
in one knowledge domain may be at veiy elementary levels in their development in another 
knowledge domain. This evidence has led to a domain-oriented view of development In this view, 
development is reflected by die degree of com{dexity and integration a person's knowledge structure 
has attained. Learning has come to be viewed as the construction of increasingly elaborated, com{dex, 
and integrated knowledge structures. 
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Development Within • Knowledge Domain 

Rctearchcn have identilied developmental p rogrci ii on in a knowledge domain in terms of tfie 
nature of concepts or constructs that appear in people's written and oral expressinns regarding 
problems or explanations. Understanding of patterns of devdopoicnt within a knowledge domain have 
come botfti from studies that either (a) compare indivMuals who are at different levcb of devdopment, 
or (b) follow ttie same individuals so^ stages of devdopment Expett-novioe studies compare 
individuals who are at different levels of devek)pment These studies have shown that experts attend 
to deeper, prindple-related features of situations and use more powerful, inclusive concepts and 
principles in their inteipretationsy while novices focus on surface features detectable throu^ ttie senses 
and are more concrete in their interpretations. Experts focus selectivdy and strategically on cues that 
are relevant to a problem, task, or goal; novices attend to a %vkler array of cues and pay as much 
attention to those that are irrelevant as to ttiose ttut are rdevant 

Other studies have focused on diaracteristics of the concepts used by people wMi high- snd low- 
domain knowledge in describing, analyzing, and ei^laining domain-relevant problems^, situations, and 
events (Langer, 1980). Individuals who possess extensive, highly devdoped knowledge use 
superordinate concepts and definitions and express analogies and links that reflect abstract 
relationdtips. People with moderate knowledge devdopment dte examples and give attributes and 
defining characteristics. People witti very little domain knowledge devdopment make links to permid 
experiences and dte concrete instances rather than concepts. Devdopmentd studies of children reved 
a similar progression in abstraction, from instances snd categories of objects and ttieir properties tfiat 
are directly p erceptible by the senses, to properties and relations that are determined by operating on 
and classifying objects (Lunzer, 1986). 

Levels of abstraction within the workplace have dso been characterized Qaques, Gibson, & Isaac, 
1978). One characterization of these levels is in terms of gods and plans (Stamp, 1978). At die lowest 
levels, gods are hot generated but plans may be developed. At a middle level, gods and plans are 
generated in terms of rules (procedures), or forms (patterns). At a high levd, new patterns of goals and 
plans are generated that reflect unique aspects of situations. 

Learning as Construction of Knowledge 

Research on learning is increasingly indicating that true understanding, which reflects higher 
levels of knowledge, is constructed rather than received directly from another person- "Wisdom can't 
be told" (Bransford, Franks, Vye, & Sherwood, 1986). People make sense of thdr environments by 
constructing meanings to which tf Uer.d. These meanings enable people to see situations in certdn 
ways and guide thdr actions. Ove ne, people develop repertoires of such constructed meanings, 
known as knowk J'ge structures. Because people's knowledge structures influence what is noticed and 
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attended to, and interpretatkxts of eiqperiencct, prior knowledge influences new learning. Knowledge 
construction during tfte learning procev reflects and occurs within ttita already present structure. This 
expUins why it is often difficult to help learners change their views and W9y% of doing things. The 
knowledge structure that learners bring with then has been 4 itabHshed over several years of 
experiences. Unless learners' educational o^eriences resuh in conatructkm of a new cog^ 
organization that integ^tes, sAodifiei^ and elaborates the oM structure within a new one, old structure. 
m likely to continue to guide ttiinUng and actions and new learning b likdy to reasain incit. 

These views of learning suggest tfuit learning asa/ be defined as changes in knowledge struchues. 
Ways in which knowledge stnicturea change have been identified by Ruaaelhart * Nonnan (1978) aa 
follows: 

1. New elements are added to oM structures but Ae oM ocganizatkxi remains intact 

1 New linkages are fonned creating larger 'chunks' in the knowl^^ 

3. New organizatfons occur which reorganize and embed previous structures wittiin new 
structures diat .lave more encompassing elements at their organizatkxial centers. 

Cognitwe leseaich has also indicated Aat, in addition to constructing and integrating knowledge 
stnictures, people also learn programs for drawing from ttieir knowledge structure in order to interpret 
and cmte actions for unique situations. In other words, they learn to compile ttieir knowledge in 
ways d\at make it applicable to differing situations (Andenon, 196% Schank, 1982). 

Factors tiiat Support Construction of Knowledge 

The first factor, fidelity, is the correspondence of ttie features in a simulated or learning situation 
to the eventual situations learners are likely to encounter (Elstein, Shulman, k Sprafka, 1978). Hdelity 
in terms of 'deep features' (features which require more superordinate, generalized, lAwtract knowledge 
to see, and can only be inferred from more specific, concrete events or objects) is especially important 
Fidelity is especially important, for example, in helping learners construct relational concepts. 
Constructing such concepts requires exposure to sihiations, problems, or tasks in which instances of ttie 
complete relationship exist Goyce & Weil, 1966). Fidelity to the context in which learners are expected 
to later construct or recognize concepts is also important If learners need to see relationships 
embedded in a context containing many irrelevant as well as relevant details, learning experiences 
should provide Aat kind of context Learning experiences fliat present only what is relevant offer no 
opportunity for learners to acquire the selection capacities ttiey need to extract tfie rdevant cues which 
signal a concept's presence in tht real worid. Furthermore, if real-worid sihiations involve detecting 
and using feedback from tht environment during tht process of solution, learning situations need to 
provide such feedback so learners can develop tht capacity to recognize and use it in monitoring flteir 
own actions. 
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The Mcond factor, visualization, focuses on visual images. Visual images appear to penetrate 
deeply into cognitive structures (Reiser, 19K). Althoug^ ttiere ^»pear to be individual diCfcrences in 
the extent to which peofit are visually-oriented in tfieir thinking, images d past e)qperiences appear to 
be especially recalls ^le and may be a major avenue throu^ which other rdevant memories are 
activated The capacity of the human perceptual system to represent laige amounts of highly complex 
information at a glance and store that information in memory witti great accuracy and durability is well 
developed (Spiro et al., 1967). Because of dus, the perceptual system is ideally suited to facilitating ttie 
representation of complexity and tfie recognition of resemblances across cases. 

Third, range and depdi of experience contribute to people's constniction of knowledge. Range of 
experience refers to the degree to which experiences vaiy. A wide range of experience provides ridi 
opportunities to compare umil'* les and differences and to extract generalizations. Constniction of 
theme-level knowledge structures is dependent on range of experience. Devdopment of tfieme-levd 
knowledge requires exposure to a wide range of experiences that differ on surface features, liut are 
si;r lar on deep-level prindples or concepts Deptti in experiences refds to tfie amount of a type of 
experience and to the level at which experiences are encountered Deeper experiendng is extended in 
time and engages many intellectual facets as well as emotions; it involves detailed examination and 
analj^s of problems and situations and is personally involving. 

Criss-crossing the landscape is an instructional prindple ttiat provides depth and range of 
experience (Spiro et al., 1988). In relation to this prindple, a knowledge domain is viewed as a 
conceptual landscape. A landscape has many sites which share some features and are different in otfier 
features. A landscape is understood by exploring it from many directions, crossing it first tfiis way ind 
then that In this approach to instruction, spedfic cases are "landscape sites* whidi have partially 
overiapping features. "Case sites" are re-visited and analyzed from a number cf different directions and 
perspectives. By repeating the presentation of the same case in relation to different concepts or 
perspectives, the multifaceted character of the case is revealed to (constructed by) tfie learner. This 
process is crudal for transfer to unfamiliar situations. It frees the learner from a rule-bound approach 
and develops multiple perspectives, points of view, and systems of classification which can 
accommodate the variability that differing contexts introduce (Spiro et aL, 1987; Spiro et al., 1988). 

Finally, mediation of learners' experiences by external sources influences the interpretations or 
meanings learners place on their experience. Mediation contributes to the form knowledge structures 
take by influendng what become the structural centers of learners' knowledge structures (Luria, 1982; 
Vygotsky, 1978). Mediation entails focuring learners' attention on certain features that are relevant to 
some goal, problem, or perspective; highlighting cues present in an environment; or providing certain 
interpretations of events, actions, and conditions. 
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Sunamry 

In summaiy, theokies which integrate knowledge, mothr&tionft, actions, and environmenta have 
particular rdevance to undentanding and affecting the lystem cf concepts and mental processes that 
direct people's invohrement in real-worid activities of interest in vocational education. Leamem' roles 
in the construction of such systems of concepts, or knowledge structures, have implications for the way 
vocational education conceives of and structures instniction. Four facton were identified as 
particularly significant in supporting learners' knowledge construction: (a) fidelity of die learning 
sihiation to tfie real worid, (b) use of learners' perceptual system through incorporation d visual 
images, (c) range and depth of learner^ experiences, and (d) mediation of learners' interpretations of 
theL' experiences. 
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»IS11«XnX3NAL DESIGN iyKX)EL 

This research sou^t to develop and test an instructional design model that would support 
learners in constructing knowledge structures that have ttie following duncteiistics and functions: 
(a) integrate features of the external world, domain-knowledge concepts and principles^ and goals, 
intentions, and actions; (b) enable sdecth^, strategic focus of attention on deep4evd features; 

(c) facilitate interpretations in terms of more powerful, superordinate^evd concepts and principle^* and 

(d) support sensithdty to subde but important variations among cases. The following sections describe 
the instructional design model and tfie prototype instructional design that was devdoped to test ttie 
model 

Devdop a Theory cf the Knowledge D(mim 

The theories outlined in Chapter 2 emphasize ttie content and function of knowledge structures. 
They assume that a Aeory of a knowledge domain is a neces**^ aspect of understandmg 
domain-rdated thinking and actions. It follows that instruction intended to afiect domain-related 
thinking and actions should begin witfi an understanding of domain knowledge and the mentsl 
process^knowledge interactions that characterize ttie domain. Tlus knowledge-domain tticory docs not 
entail outlining all of Ae specifics of domain content Rattier, it invohres devdoping an understanding 
of the mtare of the knowledge structures, particularly at the higher, more over-arching levds of 
knowledge entailed in a domain, in terms of ttieir type of content, ttieir ot gsniaalk m, md ttie nature of 
their interaction with mcnid p roc ca s ea ttiat is most critical 

Real-worid problems are often complex and 'messy.' They have irregularities and partially 
overiapping characteristics whidi defy neat boundaries and categorizations. Mudi of ttie application of 
cognitive ttteory to instruction ^as emphasized (a) well-structured domains ttiat vary uniformly and 
systematically (e.g., mattiematics); (b) domains ttiat are context-independent (e.g., generd, sdentific 
concepts); or (c) domains that are limited to a small set of concepts and goals (e.g., interpreting simfde 
sentence strings or strings of terms). A complex, ill-structured domain has many concepts ttiat are 
relevant, which interact with the spedfic context in whidi ttiey are set, and whidi have incondstent 
patterns of combinaHon across case applicaHons of ttie same type (Spiro et al, 1968). Many red-worid 
problems reflect ttiese characteristics. Domdns ttiat concern human beings, especially ttidr sodd or 
psychologicd aspects, typicdiy meet the criteria for being both complex and ill-structured Because 
prior research (Thomas, 1988a) on knowledge underiying working with children had already provided 
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a st^it on a theory of a knowledge domain, and because this domain met ttie criteria for being ccmiplex 
and ill-structured, it was chosen as the domain for development of the instructional p rotot y pe. 
Developing a theory of the knowledge domain entailed gaining an understanding of tfie 
knowledge domain as a composite of vaiying types and kvds of knowledge structures. The process of 
knowledge-domain tfieoiy developjient entailed describing the domain's task-environment structures!, 
goals, and subgoals, as well as plan-action and theme structures. This was done using a combination of 
empirical and rational approaches. Details of the p ro cesses and procedures used in devdoping a tlieoiy 
of the domain, as well as tfie domain tfieoiy that was developed, are presented in Vohime I of ttiis 
publication series (Thomas & Englund, 1989). The knowledge domain that was devdoped focused on 
supporting diildren's social development and induded ttie following: 

1. Task environment induding children, activities, and contextual physical, sodal, and time 
a^cts of Ae environment 

1 Ten duld-focused sodal-development main goals and relevant subgoals and action plans 

3. Decision rules for determining priorities among goals 

4. Two illustrative, contrasting themes 

This domain theory provided an understanding of the nature and organization of content relevant to 
each type of knowledge structure. 

The second aspect of a knowledge-domain ttieory is a description of tfie interactions t>etween 
knowledge and mental fmcess. These were identified by imposing the knowledge-structure theory on 
data that had been collected previously, which contained analyses of specific situations. Individuals at 
varying levels of educational background, experience, and expertise in working witfi children were 
asked to provide an analysis of 23 video-taped situations invdving adults wcMMng witti children. The 
analyses focused on actions the adult took, their appropriateness, and further actions that could or 
should have been taken. These analyses were content analyzed for ttie mental processes reflected and 
for the kind of knowledge structure involved in each process. Table 1 reveals a summary of the results 
of this analysis. 

Describe Development Within tiie Knowledge Domain in Terms of 
Nature of Knowledge Structures and Knowtedge-Thinking Process Irtteractions 

Developing an understanding of development within a domain of knowledge focuses on two 

aspects of the theories presented in Giapter 2: (a) person-environment interaction, and (b) the need for 

characterizing what constilutes development within a domain. Development within the knowledge 

domain of supporting children's development was accomplished by imposing more general theories of 

conceptual development (Jaques et al., 1978; Langer, 198Q; Lunzer, 1986) on the interpretations of 

situations data described above, on the knowledge structures that were identified, and by 
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Table 1 

Mental Proccas^Knowlcdy Structure Intcracdoni 



Menid pioceei 



Type of knowledge Explain 
structure 



Predict 



Evaluate 



Plans 



Goal-fdan 
structures 



Themes 



Explain how an 
action fits widUn 
a series of actions; 
explain p roblems 
that occuned in 
ttte execution of 
actions 



Explain actions in 
terms of goals; 
explain consequences 
of actions in terms 
of dieir impact 
on goals 



Explain goals in 
terms of what 
people are like- 
their natures, 
assumptions, and 
belief^ explain 
consequences of 
goals in terms of 
ttieir impact on 
themes 



Predict what 
actions wiU come 
next what even ts 
wHH occur next; 
predict pvoUcms 
that may arise in 
acthrities 



Pkedictwhat 
actions are likely 
to be used or whidi 
couki be used for a 
known goal; predict 
which actions might 
accomplish a goal; 
predict what 
problems might occur 
in using ceitain 
actions to 

accomfdish identified 
goals 

Predict what goals 
or actions are likely 
based on principles that 
are operating, what 
people are like, what 
their assumptions and 
beliefs are; fmdict what 
will happen, what 
problems may arise in 
relaticm to patterns of 
goals 



Judge the 
apptoprialencss 
oifanadton's 
sequential or other 
positkMi in rdatioG 4 to 
other action^; judge 
how wdl an action 
was executed 

Judge actions in terms 
of tfieir effectiveness 
andAxr qipropriatcness 
for accomplishing 
particular goals; 
judge consequences 
of actions in Cams 
of tfieir impact on 
goals 



Judge goals in rdation 
to each other in 
li^t of priorities, 
consistency, relevance, 
and ottier ttieme-level 
principles; evaluate 
omsequences of goals 
on themes 
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integnting patterns that had been discovered in others' research (Ouunpagne, lOopfer, k Gunstone, 
1982; Qement 198% Qement ft Broivn, 1984; Spiro et aL, 1968) into die ^thesis. The result of this 
process was a Icnowledge-domain-andiored developmental diart whidi is sumtitarized in Table 2 and 
elaborated in more detail in the Appendix. 

Table 2 

Developmental Levels Based on Relating Knowledge-Structure Theory to Knowledfe Domaii?! 



Characteristics 



Beginner 



Learner level 



Intermediate 



Advanced 



Knowledge 

structures 

available 



Orientation in 
understanding, 
explaining, and 
summarizing 
situations, tasks, 
or problems 



Avenues used to 
affect situations, 
tasks, or problems 



Possesses task- 
environment 
knowledge-terms, 
concepts, principles 

Descril>es sensory- 
apprehended events 



Recalls past 
personal experiences; 
trial and error 



Possesses plans 
for predetermined 
goals 



Applies rules; 
determines causes 
for task-environment 
conditions and people's 
action^; determines 
consequences of actions 
on task environment 
and on predetermined 
goals 



Generates plans for 
predetermined goals 



Possesses ttieme: 
whidi have been 
self-constructed 



Focuses on 
underiying bdiefs, 
assumptions, and 
pe rsp e cti ves about the 
nature and qualities 
of human beings and 
ttieir rdationshipe, and 
on implications and 
consequences of goals 
for ttiemes 

Generates tt)eme*based 
goals and relevant 
plans uniquely tailcmd 
to persons and 
situations; generates 
new themes 



Table 2 reflects differences in knowledge underiying each level of development This way of 
conceptualizing domain-related development suggests instructicmal goals that focus on develqnnent of 
the type of knowledge structure die individual has l>egun to develop or on the type of structures that 
characterize the next highest level of development 
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Identify Potentid LmnUng Enviwmimii(9) timt VfiU EmUe 
Learners to Qmstrud Ae KnowUdge Domain $t Their lad of Dev^jfmmt 

The f acton of fiddi ty , vitu«Ui«tioii, imge «id dqpth d cxpoknot;, «id mcdiilion were 

considenxi in rdation to the nature of the knowledge donain, the knowledlge rtnictiireHMntal picceM 

interactioni summarized in Table 1, and ttie devdopmental pioy^ i akm aMOMriged in Tdble 1 What 

learning eqperiences, ^nvironmenti, and media would addraa thaae fadon^ typea of knowledge^ and 

knowledge-froceaa interactions? This process pointed to the potential uaefukiesa of a coaputeriied 

video-disc4iased learning environment in enabling learners to oonstnict knowledge structures rdevant 

to tiie selected knowledge domain. 

FideBty 

Because visual and auditofy cues are often indicators of deep-level features of adult-chikl 
interaction, incorporation of video-taped, real*worid situations was seen as a way of providing fiddity 
in the learning environment Video-taped situations containing tht concepts and rdationdiips of 
interest were seen as more adequately addressing die fidelity factor if diey were actual ratfier than 
staged situations. The video camera is able to record an environment wittiout interpreting cues and 
featuies for those who will view it It should be noted, however, that even die process of recocding a 
video tape is interpretive and selective to die extent that some aspects of a situation are taped and 
some are not The video camera records (a) surface and deep features alike, (b) cues diat are more 
obvious and those that are subde without calling attention to eidier, and (c) aspecis of an environment 
that are botti relevant and irrelevant to a goal, purpose, or task. Thus, video recordings of achial 
situations leaves the job of extracting factors to leamers-a task learners must perform in real-world 
Mtuations. Furthermore, l>ecau8e human-interaction situations entail many varied and complex cues, 
they aie difficult to describe in all their richness in written- or veibal-description form. The difficulties 
in portraying environments of this ccxnplexity and variability in a form ttiat captures aU relevant 
features without pre-interpreting tiiem for die learner are lafgely avoided by die use of video-taped 
situaticms. Vor these reasons, video-taped situations of adult-diild interaction were seen as providing 
Importunities for learners to perceive situations in terms of dieir own stage of knowledge-domain 
develc^ment in much the same way as learners will do in the real worid* 

It may be fearible to achieve fidelity for less-complex environments through the use of diagrams, 
photographs, and forms of representation other than videa The important points to remember are: 

1. Rcfmsentations of environments which reflect considerable interpretation are likely to have 
lower fidelity for learners in relation to real-wortd environments and amy be undiscemiUe by 
them as the task environment that is intended in die instructicm. 

2. Representations need to contain the kind of mix of cues and features diat real-worid situations 
are likely to contain. This does not mean they must contain die exact cues. It does mean dut 
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represmtations used in instruction ihoiild contain surface and deep features, obvious and moit 
subtle cues, and rdevant and irrelevant features. 

Visualixation 

Human interaction, as it occurs in ttie real worid, portrays vivid visual images that are lost when 
these interactions are diagrammed or otherwise truislated into wywbd&c forms. Verbal descriptions of 
intenKtions can produce mental images if persons reading the descriptions have had experiences that 
enable them to mentally create die images being described. PerMxi^ask environment relationships 
suggest that there is likely to be a wide range in learners' visual images of interaction patterns if tfiey 
are required to generate diem from veri>al descriptions. For tfiese reasorair die principle of visualization 
also supported ttie use of video-taped situations in die instructional p rotot y p e 

Visual images can be provided in forms other ttum video, of course. Visual images of less 
complex and more static environments may be able to be captured in diagram^ charts, (rf)otogra{rfis, or 
other media. In choosing the form of visualization, consideration of the kinds and complexity of 
images can help in determining which forms are applicable. Images of diildren and of interaction are 
dynamic The moves and changes are among the nost important aspects to visualize. Further, deep 
features revealed by interactions of adults and children are often complex and subde. Thus, for tiiis 
domain, the ability of video tape to provide dynamic and complex images was an important 
consideration. 

Range and Depth of Experience 

The combination of video-taped situations and computer control are both important in ttte ability 
of a computerized video-disc-based learning environment to prcvide a range of different esqiieriences 
and depth with respect to any given experience. With respect to range of experience, video-taped 
situations can expose learners to many real-worid situations containing knowledge-domain 
representations that would take learners much longer to gain exposure to through real-worid 
experience. In other words, video-taped situations can concentrate a wide range of experiences in a 
compressed space of time. Computer control of the video allows the situations to be experienced in a 
variety of sequences, another aspect of range of experience. Computer-controlled video is also an 
important factor in providing for depth of experience. A computerized video-disc learning 
environment allows learners opportunities for repeated and detailed examination and analysis of 
situations because learners can control the frequency and pace of situation examination. The situation 
can be "slowed down* or "replayed" in contrast to real-worid experience which occurs at an 
uncontrolled pace and time, and may vary too greatly and too quickly to allow patterns to l>e discerned 
by learners. 
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Mediation 

Mediation hi^ig^U featvtm, dri%vt Icamcri' attention to certain patterns or occurrences, and 
may provide a lens through which to view a situation. Because computers can easily shift leamets' 
focus of attention with respect to a situation, a sing^ situation can be viewed from severs! perspectives 
and, fhm, provide consideralrie depth of experience Mediation is one way in whidi deep-levd 
features of situations can be highlighted. While SMdiation of experiences cm be provided by teachers 
or other sources external to learners, using computers to mediate learning assures tfut all learners are 
exposed to mediation. The fonn of the mediation can be varied accofding to learner needs if computer 
programs are sophisticated enough to provide for this. Mediation can be interjected at strstegic, critical 
times tfuou^KMit the learning process; these times can be detennined and controlled by either an 
instructor, learners, or the computer. 

The type of mediation provided needs to be governed by the developmental stage of learners. 
Beginning learners who are learning basic concepts and principles in a discipline, and using ttie 
concepts and principles to interpret environments, need mediation that hdpa them connect the 
concepts and principles with actual events and conditions in the external worid Mediation which 
helps l>eginning learners see below sensory-detected patterns to patterns that indicate deeper, 
principle-dependent features will help them move to a higher levd of devdopment Intermediate 
learners who are in the process of developing goals, and links between goals and plans tlut can 
accomplish the goals, need mediation that focuses their attention on goals and on the connections 
between goals and actions. Mediation for learners at this level nug^t be in the form of questions that 
guide learners to (a) look for goals underiying situations and actions, (b) lock for and predict actions 
that relate to givet^ goals, (c) explain why certain actions were done, (d) explain situations in terms of 
goal-action connections^ (e) critique actions with respect to their consequences for goals, and (f) 
compare their own explanations, predictions, and critiques with others at the same or differing learning 
levels. Advanced level learners who are in the process of developing themes and learning to generate 
goals from tiiem are supported by mediation that focuses their attention on connections between 
themes and goals. This level of mediation might take the form of asking learners (a) about similarities 
and differences in cases that differ in context but are similar in thematic prindples; (b) to explain the 
sources of goals; (c) to consider the consequences of goals for themes; (d) to predict goals of people for 
whom they have information alxHit perspectives, beliefs and orientations; (e) to critique goals in 
relation to certain themes; and (f) to compare thdr own perspectives, explanations, predictions, r id 
critiques with those of others at the same or differing learning levels. 
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Deodop a Structure VfiOun Which Strstegicalhf Mediated Lmmer-'Envmmment IfUemciion Can Occur 
The way in which leamm experience environments is the central question here. Should 
environments be grouped on the basts of goals? Task environment features? Themes? To how many 
environments must learners be ejqposed? How often should medii^ion be provided? Inrdationto 
what learner patterns? In relation to what environment patterns? These and countless odier questions 
are answered one way or the other when learning environments are structured. Because the structure 
of the learning environment depends on ttie level of ttie learner, the nature of ttie knowledge domain, 
and the nature of domain-related environments, structuring b a matter of judgment more ttian rules. 
However, some general considerations will be provided here, as well as a description of ttie q>cdfic 
structuring ttut was done in the prototype and the reasons underiying that structuring. 

Selectinf and Creating Visual Images 

Structuring a computerized, video-disc-based learning environment involved creating the video 
and designing the computer program to reflect the considerations with respect to fidelity, visualization, 
mediation, and range and depth of experience discussed atxsve. Ninety-ttuee, non-staged vignettes of 
typical situations relevant to children's social development which reflected the knowledge domain were 
video taped in actual work settings^ With the help of a panel of experienced, expert practitioners in the 
domain, each vignette was analyzed in terms of the aq^ects of ttie knowledge dcmain it reflected. 
Twenty-three vignettes were selected to represent, within ttie ^>ace limitations of a two-sided laser 
disc, as much of the knowledge domain as possible In addition, other criteria for vignette selection 
included encompassing as wide a range of situaticms as possible, including situations %vith both surface- 
and deep-feature similarities and differences, and including a range of woric-personnel-«xpertise levels. 
A master video tape was produ< containing the twenty-three vignettes. A master, level III video disc 
containing 54,000 video frames on each side was then pressed from this video tape. Each frame on the 
disc was assigned an index number. The index number allows the computer to And the location of 
specific frames on the video disc. 

Creating the Environment's Structure 

A concept of knowledge structures as momentary assemblies (Newell, 1961; Schank, 198% Spiro et 
al., 1987), rather than as static structures having rigid boundaries and content, underiies the struc;ure of 
the computer program that was developed. An analogy may be helpful here. Statistics programs thai 
run various analyses on a file of data produce graphs, charts, and numerical summaries. A graph or 
chart that appears on the screen reflects a momentaiy picture of the data that disappears when one is 
done looking ai it and closes the screen. The graph or chart no longer exists. To review the chart 
again, one simply re-runs the program on the same data flle and the same chart reappears. 
Furthermore, the same program can be run using different data sets to produce charts and graphs that 
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0uy look quite diffmnt from one data act to the The computer programa that were designed for 
tf\e beginning and intermediate leveb of ttie prototype learning envinxunent repreacnt oMxMa of tudi 
a ^knowledge latembly" program. 

Each of these programs wm constructed around a central tree* or "spine* structure to which 
lelathrdysmdl knowledge strands coukl be attadied. This allowed diffmnt central trees or spines to 
be configured to represent different organixatkms of the saaM knowirM These types of structures 
also aDowed the saiiie knowledge to be recombinedwitti different k^ For example, the central 

tree for the beginner levd program was structured on the basis of aspects of the task en vironment 
This tree was a program for combining knowledge about diiklren witti knowledge about acthrities, and 
physical and social environments. The central tree for the intermediate levd program was structured 
on Ae basis of main sodal-devekipment goals and related plans. These trees are shown in Figure 1. 
Small strands or packages of knowledge were devdoped whidi couki be linked in tfie computer 
program to Ae tree(s) in whidi ttiey were rdevant Video was nicorporated in ttte knowledge strands. 
Thus, a single knowledge padcage (and its video segments) could be used several times, but eadi use 
invohred different relationships and different purposes-Unkages to different parts of ttie knowledge 
structure. This type of program structure was seen as reflecting tfie same kind of flexibility intended as 
a characteristic of the knowledge structures learners would coiutruct tfiroug|i experiencing ttie learning 
environment 

The concept of criss-crossing the landscape was employed in tfie use of the scenes as cases on ttie 
video disc Each scene was incorporated several times, in several different knowledge strands. Eadi 
time it was induded, it was used in relation to a different concept, plan, or goal (landscape "sites"). 
Thus, the same scene was explored from many directioiu. 

Since individuals are likdy to be at varying stages ivittiin a levd of development, spedal 
structures were induded in the central trees. One of ttiese structures was referred to as a "bud;" Ae 
other was referred to as a 'tendril' Because the knowledge structures regarding ttie task environment 
ani plans, goals, and themes needed to function together as a total knowledge system ratfier dun as 
discrete, separate structures, opportunities were provided for learners to lirdc tfiem together and 
integrate them. Buds and tendrils were aimed at ttiese linkages. A bud is an introduction to ttie next 
level in relation to the current level. A bud structure wks induded in hctix the t>egirmer and 
intermediate level programs. It set the stage for the next level of development by lirUdng learning at 
the present level to learning diat was to come in the next level. 

Buds in die beginner level program began an exploration of goals and plans (intermediate level 
structures) relevant to the task-environment concept bdng exfdored. Buds in the intermediate level 
program began an exploration of themes (advanced level structures) in relation to cases already 
experienced in connection with a main goal (see Figure 1, mtermediate levd computer program 
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structure). This tfieme bud is die 'Adult Assumption; and Goals' portion of die intcmcdiatc level tree. 
The uc{riorad<H) of dteoies at this levd was leas extensive and less in depdi dum dut induded in die 



Task Environment Concepts 
and Principles 
(Beginning) 



Environment 



Children 



Introduction 



Introduction 



Activity 



Subgoals 



Plans & 
Plan 
Actions 



Goals & Plans 
(Intermediate) 



10 Main Social 
Development 
Goals 



Related Concepts 
and Principles 



Adult Assumptions 
and Goals 



"tendril" to previous level "bud" to next level 



Related 
Concepts and 
Principles 



"tendril" to previous level 



Figure 1. Structure of computer program for beginning and intermediate levels. 
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advanced level learning environment Likewise, fht exploration of gomti and plans initiated in the 
beginning level program 'Inid* was less extensive and less in depth than tfiat included in tfie 
inteimediate level learning environment 

A tendril "reaches back" into the previous level and provides a linkage between the levd 
learner has completed and the levd of the program currendy being r^perieiKed. A tendril "pulls" 
previous learning along and embeds it %vittun current learning. The intermediate level tendril that 
hooks back into the beginning level is highli^ted in Figun? 1. This tendril is represented by ttie 
"Related Concepts and Principles" portion of the intermediate levd program structure. This tendril 
routed die learner to beginning level task-environment-focused knowledge diat was relevant to die 
goal or plan being explored in the intermediate levd learning environment The primary objecuve in 
such backward routing is the linking and embedding of prior knowledge in newly devdoping and 
more advanced knowledge structures rather dian review of knowledge. Because die prior knowledge 
is experienced in a new way and in a new light, revisiting it is a aew experience rather than a review 
of an experience one has already had. 

Creating Forms Patterns of Mediation 

Once the tree structures, buds, tendrils, and knowledge strands or packages were determined, 
detailed development of the computer program could proceed. This invoh^ed creating die details of die 
tree structures, selecting and organizing content of knowledge strands, and identifying and 
inco^ grating relevant video segments. It also involv i developing die mediation to be prrdded by 
the computer as learners experienced the scenarios. 

A conscious attempt was made to vary the form of mediation provide ^ die computer so that 
learners' experiences in the learning environment did not become routmv. * ile die extent to whidi 
the video could be varied was limited (die disc only held one houi^s wordi of scenarios), die form of 
mediation had fewer limitations. Forms of mediation used induded (a) asking learners (in advance of 
watching a scene) to look for spevitic things, (b) posing open-ended questions after learners saw a 
scene, (c) combining approaches (a) and (b), (d) asking learners to till in charts ^ diey watched scenes, 
(e) having learners share completed charts with each other after a scene was completed, (f) asking 
learners to compare dieir charts witii one provided by the computer, (g) asking learners to choose 
options and dien providing feedback on their choices, (h) giving learners opportunities to practice 
certain actions at spedfic points, and (i) giving learners opportunities to determine die points at which 
certain actions should occur. 

Creating the Advanced. Theme-Development Level 

A form was created for the advanced k I of the learning environment that was quite different 
from that of the beginning and intermediate levels. The advanced level used a limited computer 
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program which functioned only to drive the video-disc player. Learning was not mediated by tfie 
computer but rattier by a series of written padcets which »ked (q;>en-ended questions whidi focused 
initially on a set of video scenes involving the same adult and then moved .o reflections on personal 
experiences and personal patterns of goals, assumptions, and beliefa. The computerized video<lisc- 
based learning environment was still important in tfiis levd because it allowed ttie video segments to 
be grouped in ways that were different from prr/ious groupingi and also gave learners control and d\e 
opportunity to explore additional groupings. In the advanced level, all ttie video segments on the 
entire disc were grouped on the basis of which adult was in the scene All segments witfi the same 
adult were grouped together to facilitate discovery of the themes underiying the adulf s pattern of 
actions across situations. The packets provided instructions and guided learners' exploration of ttie 
video segments from the perspectives of assumptions, beliefs, and priorities. Tendrils at tttis level 
reached back to the intermediate levei goal-action focus, but did so in relation to discovering or 
considering possible sources of the goals. 

A second segment of the advanced level focused on a different level and type of case analy^ 
This portion of the advanced level was intended to (a) address titt range-of-experience principle, 
(b) continue the development of general themes relevant to a wide range of cases in different contexts 
and situations in^'olving adults and children, and (c) develop 'case-based' knowledge of specific cases 
indexed by ttiemes. Neither the computer nor tfie video disc was needed for this lev el of tfie learning 
environment Instead, a series of video-taped situ^tions were identified as appropriate for exposing 
students to cases in a less-controlled order, an order that is typcal of ttie way experiences are 
encountered in the real worid and which makes detection of patterns in them so difficult when ttieme- 
level knowledge has not yet been developed. The structure of Ae entire prototype is presented in 
Figure 2, including the beginning, intermediate, and advanced levels. 

Learning Environment Hardware and Software 

An IBM InfoWindow System was used for the parts of the prototype learning environment 
requiring computer mediation and the video disc. The computer program was written in Turbo Pascal, 
Version 4, and required DOS 3.1 or higher and 640K of RAM. The program was developed for an 
InfoWindow System run by a PS/2 computer. Model 30 or higher, and is t)elieved to be also operable 
on an AT Model computer 

Review Processes and Pilot-Testing 

Plans ^jr, and development of, the prototype learning environment were reviewed regularly by a 
panel of experienced, expert practitioners in the knowledge domain. The prototype learning 
environment was pilot-tested twice as it v . being completed. The first pilot test occurred eariy in 
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Fi gure 2, St^jcture of learning environment supporting development of beginning, intermediate, and 
ad% anced knowledge structures and thinking processes. 
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development and was intended to determine learners' general reactions to the learning environment, as 
well as to obtain data that would reveal if learners at various levels could be differentiated by their 
responses to ttie scenarios. The rtsults of tfiis test indicated that the 19 learners who were involved 
reacted positively to the learning environment and quicMy adjusted to ttie technology which was new 
to almost all of them. Furthermore, the learners could be differentiated in terms of their learning levels 
on the basis of their responses to the scenarios. The results of tfiis initial test are reported in Volume I 
(Thomas & En^und, 1969). 

The second pilot test focused on the intermediate level of ttie program, which was ttte portion of 
the program that was oMnputerized first. Since this level had both tendrils reaching back to the 
beginning level and buds extending to the advanced level, it was a useful level on which to obtain a 
preliminary test of design concepts underiying all three levels of the learning environmer* and to 
provide input to the further development and computerization of diese levels. This second pilot test is 
reported in Chapter 4. This second pilot test also was a preliminary exploration of the effectiveness of 
the intermediate level of the learning environment in facilitating development of the kinds of 
knowledge structures and knowledge-thinking-process interactions described in Chapter 2, in Tables 1 
and 2, and in the Appendix. 
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CHAPTER 4 



EVALUATK)N OF INSTWJCrnONA^ WTERMEDIATE l£VEL 

A computer-based video-disc learning environment was devdoped based on the instructional 
design model descril>ed in Chapter 3. This instructional prototype was conceptualized as having three 
levels: 

1. A beginning level focused on concepts and prindides !Tgarding child development; on 
concepts and principles regarding jrfiysical, social, contextual, and acthdty aspects of task 
environments involving children; and on setting the stAge for linking these to plana and goals 
for supporting children's social development 

2. An intermediate level focused on the development of plan and goal structures, on embedding 
concepts and principles from the beginning level within these structures, and on setting the 
stage for linking goals to themes. 

3. An advanced level focused on the development of themes, case-based knowledge, and the 
formation of links between specific cases and the knowledge structures. 

The design for each of the levels was structured to criss-cross the landscape of concepts relevant to 
supporting children's social development. Knowledge strands to be embedded in the level one 
(beginner) and level two (intermediate) computer programs were identified. The computer program 
and knowledge strands for level two were developed. Level two is the portion of the instructional 
prototype that was tested in the evaluation reported here. 

Evaluation Design 

The evaluation design was developed to determine the effectiveness of the computer-based video- 
disc learning environment in facilitating development of the types of knowledge structures reflected in 
the knowledge domain (see Volume I, Thomas & Englund, 1989) and the knowledge-process 
interactions reflected in Tables 1 and 2 in this publication. While information about students' 
experiences with the learning environment relevant to revising and refining it was also obtained, that 
information is not reported here since this publication focuses on the effectiveness portion of the 
evaluation. 

Three methods of assessment were used in the effectiveness evaluation. The first was an 
assessment approach developed in an eariier phase of the Higher Order Thinking Research Program at 
the Minnesota Research and Development Center (Thomas, 1988a). This approach, known as a Tailored 
Response Test (TRT), had been designed to test the application of knowledge in making judgments 
regaiding adult actions in situations relevant to the social development of young children. The TRT 
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had been previously tested with various types of learners including hi^ school students, technical 
college students, associate degree students, baccalaureate degree students, and adult education 
students. The TRT contains nine video-t^>ed vignettes of adults and diildren, and a written paragraph 
discussing each vignette. Individuals are asked to watch each vignette and dien to edit die paragraph. 
Editing involves crossing out words, phrases, or sentences until tiie paragraph reflects the person's 
judgment about the situation. The eariier study involved 127 learners similar to diosc in die present 
study. Statistical analyses of that stud/s data revealed internal consistency coefficients on the nine 
paragraphs that ranged from .70 to .91, as well as concurrent validity data tfiat supported die TRrs 
validity (Thomas, 1988a). 

The other two assessment strategies were developed qf>ecifically for tiie learning environment 
Their use in this study constitutes a pilot-test of these approadies. The first of diese consisted of a 
diagnostic test designed to inu'rate which level of the program was mosi appropriate for a given 
learner. This test involved viewing three scenes on the video disc that were similar in their surface 
features but different in their deeper features. The similarities in surface features included die 
following: all involved (a) lunch scenes, (b) similar menus, (c) preschool children, and (d) a group of 
children and one adult sitting at a table. The deeper features which differed among the scenes 
included (a) the degree to which children were engaged as participants in lunch-related tasks such as 
getting and passing food, (b) the amount and nature of conversation among the children and between 
the adult and the children, and (c) the roles taken by the adult and the children in die Mtuation. It 
was expected that individuals at intermediate or advanced levels of development in die knowledge 
domain would detect deeper features and that individuals at a l>eginning level would focus dieir 
attention e*> surface features. Empirically determining the diagnostic-test-score levels which would 
differentiate l)eginning, intermediate, and advanced level learners was one of the goals of diis 
evaluation. In addition to detecting learners' levels at the beginning of their experience with die 
learning environment, a second function of the diagnostic test was to determine learning gains through 
pre- and post-test comparisons. Since the diagnostic test involved scenes used in the learning 
environment for other purposes, it was viewed as involving less transfer of learning dian die TRT, 
which involved scenes and people that were different from those used in the learning environment 

A ihird assessment approach, identified as scenario analysis, involved learners in watching a brief 
scene that was unfamiliar to them and writing an open-ended interpretation of the scene. Written 
scenario analyses were content analyzed for the types of knowledge structures they reflected (e.g*, 
environment, plans, goals, themes), and for the degree of development that was reflected within the 
knowledge-structure type. The latter dimension was indicated by the level of abstraction reflected in 
the concepts used and by the degree to which children or adults were the central focus according to 
the domain-development progression (reflected generally in Table 2 ^nd specifically in the Appendix). 
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The descriptions included in the Appendix were used as the tiieoiy4ML>ed coding documentition for 
this content analysis. The detail of tfie An>mdix is summarized in Take 3. Table 3 provides a 
description of tiie layers of development in relation to tiie types of knowledge structures, links among 
the types of knowledge structures were also noted in tfie content analysis and interpreted as evidence 
of integration within and among the knowledge-structure types. Links were scored according to a 
procedure developed by Novak and Gowin (1984). Witiun<dl links were assigned a score of 1 (see 
Table 3). Within-column links (links across layers %vithin the same knowledge-structure type) were 
assigned a seem of 5. Links within rows and links crossing rows and layers (links running horizontally 
and diagonally on Table 3) were assigned a score of 10. A graq^hing procedure was developed which 
provided a visual, qualitati^re picture of an individual's tiiought and knowledge. 

Procedures 

All learners who participated in the test of the learning environment were volunteers. Thirty-two 
learners of varying background within the knowledge domain experienced the learning environment 
for between 10 and 30 hours in groups of two to four people. The reasons for having learners 
experience the learning environment in groups radter dian as individuals were both theory-based and 
practical in nature. Theoretical reasons focused on the opportunities for deep processing and mediation 
of concepts and issues that discussion of tiiem is likely to provide. Deep processing occurs when more 
time is spent thinking about a concept and the concept is linked to a range of other ideas. Dialectic 
thinking occurs when one confronts differing views. The potential for mediation supportive of 
learning is especially enhanced when group members vary somewhat, but not too much, in their levels 
of development witiiin a domain. Higher-level learners are a likely source of learning-supporting 
mediation for lower-level learners. Practical reasons for grouping learners were related to access to the 
learning environment since equipment comprising only one learning environment was all that was 
available and all learners had to use this one environment Practical reasons also governed how learner 
groups were constituted. In some cases, practical considerations outweighed the mediation reasons 
described above, since individuals' schedules were a primary determinant of when they were available. 
Furthermore, some groups came from the same work environment with the intent of being grouped 
together. 

Participants in the evaluation were pre-tested during their first session and post-tested during 
their final session of participation. All three assessment materials described earlier were used as pre- 
and post-tests. The three pre- and post-assessments took apfrnxdmately 4 to 5 hours to administer. 
Consequendy, individuals whose participation was limited to 10 hours experienced approximately 5 to 
6 hours of instruction, 20-hour participants experienced approximately 15 to 16 hours of instruction, 14- 
to 16-hour participants experienced approximately 9 or 10 to 11 or 12 hours of instruction, and 30-hour 
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Tables 

Developmental Progression in the Knowledge Domain of Supporting ChUdren's Social Development 



Typtt of KnoMltdg* Structure* 



Dcwlopnnt 
lay«r 


EfwironMfit 

fffltvrfi 

(S«ttfno, 
chlldrtn, 
■ctlvltitt) 


JIdutt actlone 

(Novee of edult 
within taek 
envlrormsnt) 


(Act lone that 
can be used to 
achieve goala) 


Coala 

(Notlvea, 
Intentions) 


rriffTftlff 

( laportence of 
goela) 


(^repectlvee, 

belief a, 
aaaiJBDt 1 una ) 


1 


Sptclfic 
Inttancet of 
task'tnvlrormnt 
elements 


Speclflc-ectlon 
Inetencee 


Specific-action 

Inatance 

poaalbllltlea 


Adult- fcciiaed 

motlvea, 

Intentlona 


laportance placed 
on mdult-focuaed 
goala 


Vfewa of children, 
edulta. altuatfon In 
e epecffic context 


2 


Sense * sf)y>re(>ended 
concepts, 
categories of 
tssk-enviroTMent 
elesients 


Senee- apprehended 
action concepts, 
categories 


Action 

poaalbllltlea In 
terma of aenae- 
apprehended 
concept a, 
categorlea 


Situation- 
apecif 1c, 
chlld-focuaed 
motlvea, 
Intentlona 


Igportance placed 
on chlld-focuaed 
goala related to 
group needs, leaa 
developmental ly 
algnlflcant goala 


Vlewa of children, 
adulta, situatlona 
In general --acroaa 
contexta 


. 


Inferred, 
relstional 
concepts end 
cstegorles of 
tssk-envlronment 
elements 


Inferred action 

concepts, 

cstegorlee 


Action 

poaalbllltlea In 
terma of Inferred 
action concepta, 
categorlea 


General chlld- 
focuaed motlvea, 
Intentlona In 
terma of apecif 1c 
areaa of 
developmsnt 


liportence placed 
on chlld-focuaed 
goala related to 
Individual needa, 
developmental ly 
algnlflcant goala 


Vlewa of qualltlea 
of reletlonahlpa 
between edulta and 
children; pervaalve, 
generel quel It lea 
that character lie, 
•iamerize altuatlona 


4 


Theoretlcsl 
concepts, 
cstegoriee of 
tsek-envlrorwent 
elements 


Action complexes, 
systems 


Action 

poaalbllltlea In 
terma of action 
complexea, 
ayatama 


General chlld- 
focuaed motlvea, 
Intentlona In 
terma of broad 
areaa of 
developsient 


laportance placed 
on chlld-focuaed 
goela releted to 
broad, theoretical 
areae of 
developsient 


Vlewa of 
conaequencea or 
lapact of goala on 
chlldrenia 
developsmnt and 
learning 
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participants experienced appraxiinately 25 to 26 hours of instruction. One of Hte 30-hour groups 
experienced a few less instructional hours since tiiey provided considerable input into tiie development 
of tite intermediate level bud structure and materials. 

Findings and Discussion 

Of the Airty-thrce leamera who took ttie pre-tests, thiity-two com(rieted Ae number of hours of 
instruction they contracted for and took Hi*; post-tests. In one case, a person who began participation 
with another individual finished on her own when her partner^s sdtedule changed and tiie two could 
not continue working together. 

Profile of Participants 

A demographic profile of the participants is provided in TaUe 4 and TaUe 5. TaUe 4 reports tiie 
gender, age, and hi^est level of post-high school education com|;deted by participants in tiie learning 
environment evaluation. Most were female, between 23 and 40 years of age, and half had comfrieted 
some level of higher education. 

Table 4 



Particic^ts Completing Pre-Assessments: Gender, Age. and Highest Level of Post-Hid) Schod 


Education Completed fN - 33) 




N 




Gender Female 


30 


91 


Male 


3 


9 


Ags 17-22 


4 


12 


23-30 


7 


21 


31-40 


12 


36 


41-50 


5 


15 


51-60 


4 


12 


No response 


1 


3 


Highest Level of Post-Hieh School Education Completed 






One year post-high school 


1 


3 


Associate degree 


2 


6 


Baccalaureate degree 


6 


18 


Master's degree 


7 


21 



39 
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Tables 

Current Student Status of Participants in Lcaminy Environment Evaluation 



Current 




Level within current 




student status 


N 


student status 


N 


Not a student 


3 




— 


High school 


6* 


Senior 


5 






JunicMT 


1 


Technical college 








program student 


4» 


Second year 


1 






loUrC 


3 


Associate degree 








Dfogfam sludmt 


1 




1 
1 


Undergraduate in 


6 


Senior 


5 


baccalaureate 








degree program 




Junior 


1 


Continuing 








education 


3 






student 








Post-baccalaureate 


10 


Doctoral 


5 


program student 












Masters 


5 



* Two high school students were enrolled in technical college programs; thus, the total number of 
technical college students was six rather than four. 



Table 5 indicates that most participants were students enrolled in an educational program of 
some type. The types of programs in which participants were enrolled varied considerably, from a hig|i 
school program to a doctoral program. The distribution of individuals across these various educational 
levels is shown in Table 5. 

The majors of individuals who were enrolled in postsecondaiy programs are reported in 

Table 6. 
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Table 6 

Majors in Past and Present Post-Secondary Programs of Enrdlment by Participants 



Major N 



Major in program of current enrollment 

Technical college program 

Child development 4 

Associate degree program 

Human services (child dev. concentration) 1 

Baccalaureate program 

Child psychology 3 

Home economics education 3 

Math 1 

Sociology 1 

Mastei^s program 

Home economics education 5 

Early education/Special education 1 

Doctoral program 

Family life education 4 

Ministry, church growth 1 

Major in previously completed programfe^ 

Two-year post-high school program (nondegree) 

Bible 1 

Baccalaureate program 

Child psychology 1 

Elementary education 2 

FamUy relationships 1 

Home economics education 1 

Physical education 1 

Psychology 2 

Social sciences 1 

Master^s progiw . 

Home economic^ome economics education 6 

Social Work 1 

Theology 1 
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Pre- Assessments 

One question which existed concerning the three af^maches to assessment was die degree to 
which they overlapped, or measured the same ttiing, and ttie degree to which ttiey measured different 
constructs. Furthermore, since learners had widely vaiying levels of educaUonal background and 
experience with children, some were likely to be l>eginners, some intermediates, and some advanced 
learners in the knowledge domain. If this was the case, did ttie assessments opmtt differently for 
these groups? Was the diagnostic able to identify individuals of differing levels of domain-knowledge 
development? Table 7 presents pre-test score data on the three assessments for all participants who 
completed both the pre- and po^Nmeasures. 

Table 7 

Pre-Test Scores on Tailored Response Test Diagnostic Test, and Scenario Analysis Links 

Test N Minimum Maximum Mean Median S.D. 

TRT 32 140 249 195 190 26.8 

Diagnostic 32 -5 15 6 6 55 
Scenario 

analvsis links 31 10 166 56 51 35.2 



The scores on both the TRT and the diagnostic were closer to the top than to ttie bottom of die 
possible ranges. The possible range in scores on the TRT was 0 to 254. Consequendy, a score range 
from 140 to 249 indicates diat individuals scored in the upper half of ttie possible score r&nge and tttat 
some were very close to a perfect score on this lest The mean of 195 and median of 190 further 
suggest that this group scored in the upper ranges of the possible scores on this test These score 
patterns introduce the possibility that score ceilings may have been operating on this test for some 
members of the group. Similar patterns are suggested in die diagnostic test data. This test had a 
possible range from a low of -16 to a high of +18. High scores of 15 and a mean and median of 6 
points suggest that the group scored toward the hagh end of the possible range. Since the scenario 
analysis was open-ended, there was no ceiling for this assessment 
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Possible relationships between the three pre-tcst assessments were investigated by correlating Ae 
pre-scores. Since the number of participants was small, correladons should be Interpreted with caution. 
The resulting coefficients are reported in Table 8. 



Table 8 



Interct^.^elations Among Pre-Test Scores 



Prc-test 



TRT 
(N »32) 



Diagnostic 
(N -32) 



Scenario analysis links 
(N.31) 



TRT 



1.00 



-0.02 



033 



Diagnostic 



1.00 



-0.12 



Scenario analysis links 



1.00 



The coefficients in Table 8 indicate that scores on ttie diagnostic are unrelated to TRT scores and 
reveal a low, negative correlation %vith the scenario-analysis-links scores. The TRT and ttie scenario- 
analysis-links score, however, bear a moderate, positive relationship to each ottier, whidi is significant 
at Ae .10, but not at the .05 level (df » 29). Thus, it would seem that the diagnostic and ttie TRT are 
tapping different constructs and that there may be some overiap in ttie constructs measured by ttie TRT 
and the scenario-analysis-links scores. 

In addition to the links scores, a second type of analysis was applied to the scenario-analysis data. 
This analysis examined the types of knowledge smictures reflected in Ae written analyses of the video- 
taped scene. Table 3 was applied to Ae written analyses as a grid. The content of eadi written 
analysis was segmented and coded wung tfie documentation in tht Appendbc A segment represented a 
thought unit categorizable within the knowledge structure ^^pes and layers represented in Table 3 and 
described in more detail in the Appendix. Selected diagram > developed during the analysis arc 
presented in Figures 3, 5, and 7 and provide a sense of the qualitative nature of this data. Each point 
in these figures represents a thought-unit segment as defined above. The types of links described in 
the evaluation design section of this chapter in connection with their scoring are represented in the 
diagrams according to the legend provided with the figures. 
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Fi gure 3 . Scenario Analysis - Pre-assessment: Learner A. 
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Fi gure 4 . Scenario Analysis - Post-assessment: Learner A. 
Le gend 

thought unit 

links within knowledge-structure type and layer 

Jinks within knowledge-structure e (columns) and across layers 

— links across knowledge-structure type (columns) 
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Fi gure 5 . Scenario Analysis - Pre-asscssment: Learner B. 
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Fi gure 6 . Scenario Analysis - Post-assessment Learner B. 
Le gend 

thought unit 

links within knowledge-structure type and layer 

links within knowledge-structure type (columns) and across layers 

links across knowledge-structure type (columns) 
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Fi gure 7 . Scenario Analysis - Pre-assessment: Learner C. 
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Fi gure 8 . Scenario Analysis - Post-assessment: Learner C. 
Le gend 

thought unit 

links within knowledge-structure type and layer 

links within knowledge-structure type (columns) and across layers 

— links across knowledge-structure type (columns) 
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The independence of the TRT and the diagnostic reflected in Table 8 is useful information. The 
TRT was developed to assess the application of knowledge in making judgments about the actions 
people take. TRT scores are believed to reflect degree of knowledge development within ttie 
knowledge domain of supporting children's social development and ability to connect domain 
knowledge to speciflc cases. The diagnostic is a simpler and more straightforward device intended to 
tap the degree to which a person's focus is limited to surfice-level sensory information or extends to 
deeper features that require more knowledge and interpretaticm to detect Even though the total 
distribut^cms for these two measures appear to be independent, it is possible that a curvilinear 
relatiostship may exist in which poritive relationships exist at some points cm the distributions and 
negative relationships exist at other points. The possibility also exists that the scenario-analysis-links 
scores may also l^e related to the diagnostic in a curvilinear fashion. The present data are too limited to 
allow the kind of exploration that could shed light on these possibilities. 

The moderate, but not quite signiflcant, relationship between the TRT and the scenario-analysis- 
links scores suggests that a significant relationship might be found with a larger number of 
participants. Furthermore, these data suggest that the TRT may tap knowledge development and 
integration across the spectrum of knowledge-structure types, a function for which it was intended 
when it was developed. 

Comparison of Pre- and Post-Assessments 

After instruction, the same assessments that were administered as pre-assessments were 
re-administered. Differences in pre- and post-assessment means were compared using the t-test for 
correlated data. Means for pre* and post*assessments are presented in Table 9 along with t-test values 
and one-tailed ^ values. 

The data in Table 9 indicate significant increases in the means on the TRT and the diagnostic. 
The scenario-analysis-links score means increased, but the pre- and post-differences did not attain a 
high enough probability level to be considered beyond the realm of chance. This may be due to the 
exploratory nature of this measure and a need for additional development. Patterns in the data 
presented in Table 9 suggest that learners' abilities to see more deeply into situations, to see their deep 
features and to apply more domain-relevant knowledge in making judgments about situations increased 
from pre- to post-testing. While the learning environment likely had a significant role in these 
changes, it should be noted that, for some of the learners, other relevant learning experiences were also 
occurring outside the learning environment. For example, several learners were enrolled in child 
development, parenting, or early-educf Hon classes during the period in which they experienced the 
learning environment. It was not possible to isolate the possible effects of such other experiences and 
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Table 9 



Comparison of Pre- and Post-Assessment S core Means on Tailored Response Test. DiagnosHc. and 
Scenario Analysis Unks 



Test 


Number of 

paired 
observations 


Mean 


9. U. 


t 


iJne- 
t&iled 

e 


ul 


TRT 


32 












Pre 

Post 




195.0 
203.0 


26.8 
24.2 


-Z14 


.025 


31 


Diagnostic 


32 












Prt 

Post 




63 
93 


5.5 
7.0 


-2.68 


.010 


31 


Scenario 
analysis links 


31 












Pre 

Post 




56.0 
66.0 


35.2 
55.3 


-1.30 


.10 


30 



variables that may also have affected assessment scores. An experimental design would be needed in 
order to obtain such information. 

Comparison of Pre- and Post-Assessments by Instructional Hours 

Table 10 reports pre- and post-means for subgroups experiencing different numbers of 
instructional hours. Differences in these means were not tested for significance because of the smdl 
numbers within each subgroup. 

Table 10 indicates that, in general, means on assessmerics increased from pre- to post- 
administrations for the subgroups. There were three exceptions to this pattern. For the TRT, the 10- 
hour group's post-mean remained virtually unchanged compared to the pre-mean. Interestingly, he 
range in TRT scores for this group also remained very stable from pie- to post-testing. For the 
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Table 10 



Comparison of Pre- and PosNMeans on TRT. Diagnostic, and ScenarioAna lvsis-Links Scores by 
Number of Instructional Hours 



Test and Number of Pre Post 

hours of paired 

instruction observations Mean Range S. D. Mean Range S. D. 



TRT 



30hrs 


13 


1% 


151-249 


29.2 


207 


173-246 


21.9 


20hni 


4 


204 


181-225 


22.5 


215 


183-236 


22.6 


14-16 hrs 


5 


186 


140-224 


35.1 


197 


137-222 


34.6 


10 hrs 


10 


195 


164-246 


23.0 


1% 


164-240 


22.8 


Diagnostic 
















30 hrs 


13 


5.8 


-5 - 12 


5.0 


8.5 


0-17 


6.3 


20 hrs 


4 


9.3 


4-14 


5.5 


14.8 


12-17 


16 


14-16 hrs 


5 


6.8 


-3 - 11 


5.8 


6.4 


-5-16 


8.9 


10 hre 


10 


5.6 


-3-15 


6.5 


10.1 


-4-17 


7.5 


Scenario analysis links 
















30 hrs 


12 


59.6 


10-166 


43.8 


54.9 


0-168 


54.1 


20 hrs 


3 


45.5 


11-75 


313 


92.8 


40-139 


41.4 


14-16 hrs 


4 


38.6 


20-75 


22.3 


51.6 


20-90 


26.2 


10 hrs 


10 


65.1 


20-110 


29.8 


80.5 


15-260 


70.0 



diagnostic, the post-mean tor the 14- to 16-hour group went down very slightly. The range on the 
diagnostic widened from pre- to post-testing for this group, especially at the upper end. On the 
scenario analysis, the 30-hour-group mean went down slightly. The range widened at the lower end. 
The other nine means increased from the pre- to the post-assessment 
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A second aspect of Table 10 is of interest This is the pattern of score ranges for the various 
instructional hour groups. For the most part, each instructional-hour subgroup included individuals 
ranging widely in their pre- and post-scores. This would suggest that eadt instructional-hours group 
included individuals who were at a level of development appropriate for the intermediate level <rf 
instruction they received, as well as individuals who were beyond the intermediate level and those 
who were not yet at the intermediate level. The score pattern for the 2C^hour group varied sufficiently 
from that for the other groups to suggest that l^iese individuals may, as a group, have been most 
optimally matched in their development to the level of instruction. On the TRT pre-assessment, this 
group's score range was the narrowest of all the groups. This was also true for their post-TRT scores. 
Both their pre- and post-TRT means were the highest of all the groups. On the diagnostic, the 20-hour 
group had the highest pre-mean and the highest post-mean, and made the lafgest gain. The pattern for 
the scenario-links scores revealed that this group also had the largest gain of all the groups on that 
iicsessment. 

The small numbers in the 20-hour and 14- to 16-hour instructional groups make it difficult to 
draw conclusive inferences about patterns in the data. However, if the 20-hour group was comprised 
of individuals whose developmental levels were especially appropriate for the intermediate level of 
instruction, it would be reasonable to hypothesize that pre-diagnostic scores reflected by this group (a 
range of 4 to 14) and TRT pre-scores attained by this group (an approximate range of 180 to 225) 
indkat;: a level of development appropriate for the intermediate level learning envirc^iment To test 
the plausibility of this hypothesis M individuals whose pre-scores fell in these ranges were identified. 
Using a one-way ANOVA with instructional hours as the factor, score-change means on the diagnostic 
and the TRT for each instructional hours group were compared. While none of the f values attained 
significance, those closest to doing so were attained for the diagnostic when individuals having 
pre-diagnostic scores between 3 or 4 and 14 or 15 were included in the analysis. The results of this 
analysis for the TRT did not disconfirm the suggested range of 180 to 220 on the pre-TRT as the band 
appropriate for intermediate level instruction, but the f values were less strong than for the diagnostic. 

The data set is too small when factored into instructional-hours groups to be more than 
suggestive, but these analyses do offer helpful clues that can be further explored. The data patterns in 
Table 10 are suggestive of the importance of being able to identify the developmental level of learners 
with respect to the knowledge domain so that an appropriate level of instruction can be provided. 1 he 
number of hours of instruction was not, in itself, predictive of learning. When the analysis of 
instructional hours was conducted within pre-score assessment ranges as described above, the 
relationships between instructional hours and score changes became much stronger. 

From a theoretical standpoint, the match between the learner's developmental level and the level 
of instruction would be expected to be a strong factor in predicting the amount of learning. However, 
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there is an altcmarive cxplanarion for the patterns that arc reflected in Table 10. The 20-hour group, as 
a very small group, may be more homogeneous and thus, individuals in it may be more similarly 
affected by the learning environment The oAer, lai^, more diverse groups may not exhibit such 
patterns because of their diversity. This explanation may not entirely account for the patterns however. 
A similarly smaU gr ip having a few less hours of instruction (14 to 16 hours) did not display the 
distinctively different patterns that the 20-hour group did The score patterns in this second small 
group were more like those in the larger groups than they were like the 20-hour-group's patterns. 
While the 20-hour-group's hcnnogeneity may seem important on the surface, it is more likely that it is 
the match between the 20-hour-group's level of development and the intermediate levd learning 
environment that is the crucial factor. In other words, had the 20-hour group been homogeneous in 
pre-scores, but fallen at the extreme-low or extreme-high ends of the pre-score ranges, it is likely that 
their patterns would not have reflected the increases apparent in Table 10. 

Comparison of Qualitative Graphs 

Post-data from the part of the scenario analysis that involved identifying and graphing 
knowledge structures and links among them represented in the written expressions are reflected for 
three selected individuals in Figures 4, 6, and 8 (see the documentation in the Appendix and note Table 
3, which summarizes the Appendix). Comparison of post-graphs in Figures 4, 6, and 8 with the 
pre-graphs of the same individuals (Figures 3, 5, and 7, respectively) reveals an increase in ^a) thought 
involving more advanced layers of development; (b) thought involving plans, goals and/or themes; and 
(c) expressed links among thoughts. Since the graphing procedure was newly developed for this test 
phase, the main focus of the analysis was on whether or not it revealed and aided understanding of 
the nature of cognitive changes that took place for individuals during the period of instruction. The 
graphs included in Figures 3 through 8 were judged to meet these goals. Since the graphs were 
intended as individual rather than group data, they were not summarized beyond the links scoring that 
has already been reported. 

Summanf 

The data reported and discussed in this chapter indicate change in pre- and post-assessments that 
reflect learning gains as learning was defined in this project. Learning in anticipated directions was 
reflected in both quantitative and qualitative assessments. Patterns of scores on the TRT and the 
diagnostic appear to be independent, suggesting that they are measuring different kinds of learning. 
The scenario analysis may tap yet additional dimensions of learning, dimensions that the data suggest 
are likely more similar to dimensions reflected by the TRT than those assessed by the diagnostic. It 
was also apparent that not all individuals' scores increased. It is hypothesized that those who did learn 
were at le/els in their own development within the knowledge domain that matched the intermediate 
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level of the learning environment, and that those whose scores and graphs did not reflect learning 
were at levels of development above and below an intermediate level. 
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CHAPTER 5 



SUMMARY. CONCLUSIONS, AND RECOMMENDATIONS 

Summary 

History and Context Underiying fte Research 

The researdi reported in this document comprises a portion of tht ttiird phase of ti\€ Higher 
Order Thinking Research Program in fht Minnesota Researdi and Development Center. The first 
phase in this program of research entailed developing an understanding of knowledge and ttiinking 
processes underiying competent practice in two contexts of interest in vocational education: Work 
contexts and family contexts. The second phase of research developed an approach to assessing 
application of knowledge to work situations and knowledgc-thinking-process interactions. The 
present third phase is focused on developing instruction to facilitate higher levels of ttiinking and to 
support learners in constructing knowledge that underlies such thinking. This third phase 
incorporated, built on, and extended research completed in phases one and two. 

Puqx>se 

The purpose of this research was to apply cognitive theory and research to the development of an 
instructional design model for vocational education that would address a range of changing conditions 
in the workplace and the family. More specifically, the intent was to develop an instructional design 
model that would (a) reflect the interests of vocational education in real-world work and family 
contexts and activities; (b) focus attention on the mental aspects underlying work or family activities; 

(c) address learning goals that include, but extend beyond, performance to underlying meanings; 

(d) reduce the separation of cognition from affective and psychomotor aspects of human activities; 

(e) prepare learners for the world as it currently exists, as well as the yet-to-be-determined world of the 
future; and (f) be reasonably practical for vocational education to implement in terms of effort and cost. 

Objectives 

Specific objectives of the research reported here were to (a) develop an instructional design model 
for teaching mental aspects underlying work and family activities in a context of rapid change and high 
technology, (b) test the instructional design model by developing a prototype instructional design for 
teaching mental aspects underlying activities within a domain relevant to vocational educaHon, (c) 
develop materials required by the prototype instructional design, and (d) test the prototype 
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instructional design for effectiveness in facilitating development of the mental aspects underlying 
activities within the selected domain. 

Procedures 

This research sought to develop and test an instructional design model to support learners' 
construction of knowledge. Four factors were identified as particularly significant in supporting 
learners' knowledge construction: (a) fidelity of the learning situaticm to ttie real worid, (b) 
visualization, (c) range and depth of learners' experiences, and (d) mediation of learners' interpretations 
of their experiences. It was intended that knowledge constructed by learners have ttte following 
characteristics and functions: The ability to (a) integrate goals, intentions, and actions witti features of 
the external worid and domain-knowledge concepts and principles; (b) focus attention on deep-level 
features; (c) facilitate interpretations in terms of powerful, supcroidinate-level concepts and principles; 
and (d) support sensitivity to subtle but important variations among cases. 

The instructional design model that was developed is presented in Chapter 3. A computer-based, 
video-disc learning environment was developed as a prototype instructional design based on the 
instructional design model in order to test the model. This instructional prototype was conceptualized 
as having three levels: (a) a beginning level focused on concepts and princifdes regarding task 
environments relevant to an identified knowledge domain; (b) an intermediate level focused on the 
development of goals and plans, and on embedding concepts and principles from the beginning level 
within them; and (c) an advanced level focused on the development of themes and case-based 
knowledge. 

The intermediate level of the instructional prototype was tested in the evaluation reported here. 
This evaluation comprised the second piiot-test of the instructional prototype. The first, smaller-scale, 
initial tryout pilot test involved 19 learners and is reported in Volume I (Thomas & Englund, 1989). A 
purpose of the present evaluation, which involved 32 learners, was to determine the effectiveness of 
the intermediate level of the learning environment in facilitating the kinds of development described in 
Chapter 2, Chapter 3 (Tables 1 and 2), and in the Appendix. 

Conclwiions and Recommendations 

Effectiveness 

Findings supported the effectiveness of the learning environment in elaborating, modifying, and 
reorganizing knowledge structures in ways that enhance capacities to detect deep-level features and to 
apply knowledge in making judgments regarding actions that fit a situation. This study's findings 
suggest that a computerized video-disc-based learning environment can facilitate higher lev ds of 
cognitive development if it is grounded in theories that focus on the nature of higher levels of thinking 
and on the nature of knowledge that supports such thinking. 
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The data suggest that, as anticipated, the intermediate level learning environment was most 
effective for individuals whose level of prior knowledge development was commensurate with that 
level of instruction. Although the participants involved in this study represented a wide range of 
levels of development in the knowledge domain, all experienced ttie level two, or intermediate, 
program. Thus, the group whose knowledge was at an appropriate level for the program was a subset 
of the total group, a subset that was too small for some types of data analysis that would have been 
helpful. 

Assessments 

The tryout of the diagnostic with a wide-ranging, mixed type of group was a necessary st^r^ in 
determining how it might work as a selection device in relation to learning-environment levek and as a 
measure of learning. The data from this investigation of ttie diagnostic will be used in making 
decisions about which level of the learning environment should be experienced by individuals on the 
basis of their diagnostic scores. 

Both the TRT and the diagnostic appeared to be able to detect changes in capacities of learners 
who experienced the learning environment. While the role of the scenario analysis in relation to Aese 
other two measures is not fully defined at this point, the qualitative information in the gr^>hs appears 
to provide helpful information about changes in specific characteristics of individuals' kkiowledge The 
pre- and post-graphs of individuals' expressions contained in Aeir analjrsis of a video-taped scene 
provide a visual picture of knowledge elaboration, modification, and reorganization. Sudi information 
is potentially useful in providing possible explanations regarding score patterns produced by other 
measures, especially the TRT, and in understanding the qualitative nature of cognitive changes that 
may occur as a consequence of experience with the learning environment. The graphing technique 
captured a richness in the data that makes it a promising approach to characterizing and understanding 
cognitive changes that more traditional, objective-test-type approaches to assessing learning have not 
addressed. Efforts to further refine the approach should be undertaken. 

A crucial aspect of the design of the learning environment was the understanding and model of 
the knowledge domain that is reflected in Volume I of this series (Thomas & Englund, 1989). In the 
present publication. Table 3 (in Chapter 4) and the Appendix represent extensions of the knowledge 
domain that was reported in Volume I. The scenario-analysis graphs provide a visual picture of 
learners' knowledge construction in this domain-of putting together, assembling, and connecting 
environment, plan, goal, and theme knowledge structures and recombining knowledge to enable 
interpretation of a situation. Knowledge construction was described in Chapter 2 as the addition of 
new elements to old structures, the formation of new linkages that create larger chunks in a knowledge 
structure, and new organizations of knowledge that are more encompassing. The pre- and post-graphs 
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depicted in Figures 3 through 8 provide a visual picture of these aspects of knowledge construction. 
The graphing technique, with further refinements, may provide helpful infcnnation about ttie nature 
and results of knowl^ge construction by learners ttiat will aid understanding of ttie nature and 
process of knowledge construction at different developmental levels within a domain. The technique 
provides an approach by which ttie knowledge-process interactions outlined in TaUe 1, and ttie 
developmental progression outlined in Table Zr can be made explicit and domain specific Such 
information would provide a useful basis for ttie design of instruction. 

Further Research 

Further exploration of what appears to be independence of the TRT and ttte diagnostic is needed. 
A more explicit understanding of ttie constructs measured by each, and why tttey are independent, is 
needed. There is a possibility that tfiey may not be independent but that a curvilinear relationship may 
exist between them. This possibility should be explored* Constructs measured by both ttte TRT and 
the diagnostic appear to be addressed by the learning environment as indicated by the significant 
increases in the post-scores compared to the pre-scores. This assumes that the seem changes are due to 
experience with the computer-based video-disc learning environment and not to some other 
experiences students were also having at the time Inclusion of a control group in the evaluation of the 
learning environment is needed to test this assumption. 

In a general way, pre- to post-score increases support the theoretical dimen«ons tttat undeilie tfte 
instructional design: (a) fidelity, (b) visualization, (c) wide-ranging and deep-level e)q>eiiences, and (d) 
strategic mediation. The data presented in the inve^gation reported here portray a promising picture 
regarding the theories underlying the instructional design model and ttie capacity of tttt 
computer-based video-disc learning environment prototype to facilitate the de\elopment of mental 
aspects underlying complex work activities. More specific investigation of the relative contribution of 
each of these dimensions would be helpful in refining the learning environment and in drawing 
conclusions about the merits of each dimension. The TRT provides some me&sure of the effectiveness 
of the fidelity dimension since this assessment confronts learners with the cokiplexities of situations as 
they occur in the real world. EvaluaHon of the visualization dimension would require a comparaHve 
investigation that examined the video-disc environment in relaHon to si^lar instruction that uses 
written case studies or other forms of ca&e presentation that do not include visual images. The capacity 
of the design to provide a range of experiences, deep processing, and mediaHon that supports learning 
will t>e better understood when the data that were collected during learners' interaction with the 
learning environment and with each other have been analyzed. These data inciude sequences learners 
took through the learning environment, Hme learners spent on various portions of the learning 
environment, conversations within the small learner groups, and journal entries concerning personal 
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reactions to Ac learning environment These learning-process data will allow examination of the 
theofetical dimen«uons in terms of learner groupings and the types of mediation learners actually 
experienced. Future studies should ex^' line the interaction of the dimensions as a potential factor in 
their consequences for learning. 

The notion of criss-crossing the landscape described in Chapters 2 and 3, and ttie effectiveness of 
buds in setting Ae stage for higher level learning and of tendrils in reaching bade to connect new with 
prior learning, %yiU be better investigated when all ttiree icvels of die learning environment are in 
place. Future tests of die leariiing «i)vironment should evaluate all three levds once dtey have been 
completed and should involve a suffident number of partidpants at each level so dtat patterns diat 
may be present are revealed. 

Because die theoretical importance of learners' developmental levels widtin d\e knowledge 
domain in relation to die levels of die learning environment appeared to have empirical support, 
learner groufMngis should be more strategic and based on criteria relevant to learning, radier dian on 
schedules of individuals. However, despite the desirability of strategic learner grouping, it would be 
unrealistic to assume that the practical factors which governed much of die grouping of learners in this 
investigation can be completely ignored. Some balance between practical and substantive factors needs 
to be achieved. 

Data produced in this study suggest that 10 hours of instruction is likely insuffident to have 
much impact on learning, at least on learning of the type measured by the TRT. Some learning did 
seem to occur for the 10-hour group as measured by the diagnostic A problem with die 10-hour 
structure is that only about half of the 10 hours are devoted to instruction because the pre- and 
post-assessments consume 4 to 5 hours. Future tests of the learning environment should involve 
learners in a minimum of 20 hours of instruction (including assessments) in order to achieve a broader 
range of learning than that reflected on the diagnostic only. 

Future plans for continued work on the learning environment include (a) analysis of the learning- 
process data described above, (b) detailed development of the beg;inning and advanced levels of the 
instructional prototype, (c) computer programming for these levels, and (d) an evaluation similar in 
scope to that reported in this publication but involving all three levels of the learning environment and 
matching of learner-developmental levels with appropriate Icvels of the learning environment. The 
experience and data from the evaluation reported here will provide a useful t>asis for these future 
activities. 
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KNOWI^DGE STRUCrrURE DEFV^^ 

SUPPOFHING CHILDREN'S SOCIAL DEVELOPMENf 

Each type of knowledge structure is defined and described in tfte sections below. For each type 
of knowledge stricture, its developmental progression is characterized in terms of layers, with Layer 1 
characterizing beginning levels of development and Layer 4 characterizing tfte most advance levels of 
development For each layer, examples of expressions titat characterize Aat layer of development are 
indicated. 

Task Emrirmment 

Definition : Phenomena in the external environment as represented in the individual's perception of 
the environment. Descriptions of states, features, or occurrences in the environment 
Content FdcDtt : Children in the situation, the physical setting, objects, activities, events, and their 
physical features, attributes, or observed states; actions of children; perceived changes in environment 
elements, physical features of adults in the situation. 

Layer 1 . Directly observed, specific, concrete instances or examples of persons or objects, 
specific attributes, and features. 

Illustrations: Helena (referring to a specific person in the situation) 

The blocks in the comer (referring to specific blocks in a specific location) 
Mary said, "I'm hungry" (what someone actually said) 

Dan tied his own shoe and then tied Tomm/s shoe (a specific action taken by 
someone stated in concrete, rather than categorical, terms) 

The dog brought a block to Diane 

One child said, "I don'i want to . . . " 

Layer 1 Categories that can be apprehended bv the senses; observed directly; narrow in scope; 
few concepts subsumed under it (Main difference between this layer and Layer 1 is that Layer 
1 is not a category, but a specific instan*^^' Layer 2 is a category.) 
Illustrations: Red trucks and blue trucks 

Dolls With eyes that close 

Crackers 

Clean up 
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Appendix (continued) 



Lunch 
Slow, fast 

Children, a couple diildren, some of the children, all tfie children, the children 
that weren't helping 

Layer 3 . Inferred categories and relationships: cannot be directly observed except through a 
narrower categor}^; cover a limited range of narrower categories. 
Illustrations: Task 

Accomplishments 

Held back 

A cooperative effort 

Contributed to the cooperation 

Woriced hard 

Age level 

Help 

Involved 

Participating, participated 

Layer 4 . Idealized, inferred categories: theoretical concepts; do not exist in actuality; are 
inferred from other inferred concepts; cover a wide range of narrower categories; abstract 
concepts and principles; very broad scope (subsumes many concepts); hypothetical possibilities 
or opportunities for activities, occurrences, features, attributes, or states or for actions of 
children that existed within the environment but that may or may not have been used or 
enacted. 

Illustrations: Norms 

Sanctions 

Conceptual structure 
Temperament 
Initiative 
Industry 

The children's opportunity to think about their participation or role in the task 
was limited 
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Adult Actions 

Definition : Moves made by an agent in the task environment; what the adult in a situation with 
children did. 

Content Focus: Descriptions or reports of adult actions that occurred in terms of their features, 
qualities, or attributes; timing; how they were carried out; evaluative statements about how well an 
action was carried out. 

Layer 1 . Concrete instances of acts or actions the adult engaged in; what the adult said (direct 
quotes). 

Illustrations: She zipped Jake's jacket 

He said, "If s okay; I know you don't feel well today." 

Layer 2 : Actions of the adult or attributes of the actions that specifically occurred in this 
situation in terms of concrete, sense-apprehended categories that can be directly observed 
Illustrations: The teacher mentioned specific jobs 
The adult used the children's names 

The adult talked about duties; talked about what was happening 
The teacher got do%vn to the children's level 
The teacher stayed with them during the clean up 
The adult gave specific requests 

Layer 3 . Actions of the adult or attributes of the actions in terms of inferred categories that 
cannot be directly observed except through a narrower category; cover a limited range of 
narrower categories. 

Illustrations: He encouraged her, encouraged participation 
He offered his assistance 
She was directive, supportive, and positive 
Ht forewarned that time would soon be up 
He modeled what he was trying to help them understand 
She permitted a variety of ways of accomplishing the task 
He interacted with the children 
He participated with the children 
She acknowledged feelings 
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Appendix (continued) 

Praised 

He structured the activity 

She directed them during the clean up 

He ascribed good intentions to the children 

Layer 4 . Idealized, inferred-action categories or systems; inferred from other inferred-action 
categories; do not exist in actuality; cover a wide range of narrower categories; highly abstract 
actions. 

Illustrations: He trusted her to follow through 
He respected her wishes 

The teacher appealed to their sense of independence 

Plans 

Definition : Generalized adult actions or action categories stated as actions that could or should be 
used to accomplish a (rfan or a goal; actions that are or were good or not good to do; that should or 
should not be or have been done. 

Content Pocug Focus is on actions that might occur, what actions are recommended; questioning of 
actions taken or not taken by the adult; planned or hypothetical actions; if this condition, then tfiis 
action; evaluation of actions in terms of their appropriateness for a situation or goal; their effectiveness 
in accomplishing a goal; statements that point out opportunities for adult actions; whether or not 
opportunities for an appropriate action were taken or missed; opportunities and difficulties often 
encountered in implementing actions; identification of ways to adjust actions (this action would have 
been better than tnat action). 

Layer 1 . Concrete instances of acts or actions the adult might engage in, mi^t have taken; 

actions that are possible, that could have or should have t>een engaged in or takea 

Illustrations: She could have let Jake zip his own jacket 
He shouldn't have said, 'Ifs too hard." 
He could have sung that so and-so is a very good helper 
She should have said, *Put your hats on today because it is cold.* 
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Layer 2 . Acts or actions the adult might engage in, might have taken, that arc possible, that 
could or should have been engaged in or taken; attributes of the actions in terms of concrete, 
sense-apprehended categories that can be direcdy observed. 

Illustrations: He could have told the children in advance that it was clean-up time in five 
minutes 

He could have used a song about picking up 

It would have been better to have groups of two or three children working on 
the clay 

It might have been better to point out a chore she could do 

Layer 3 . Acts or actions the adult might engage in, mig^t have taken, that are possible, that 
could or should have engaged in or taken; attributes of the actions in terms of inferred 
categories that cannot be directly observed except through a narrower category; cover a limited 
range of narrower categories. 

Illustrations: I wonder about giving so many instructions to children 
She should interact more with the children 
He didn't push her to do more than she could handle 

Layer 4 . Idealized, inferred-action categories or systems the adult might engage in, might use, 
that are possible, that could or should have engaged in or taken; inferred from other inferred 
action categories; do not exist in actuality but only through other infened action categories; 
cover a wide range of narrower categories; highly abstract actions; attributes of actions in terms 
of inferred action categories or systems; actions referred to as a complex of actions (pervasive 
actions that summarize the situation or ail-the-more specific actions taken by the adult). 
Illustrations: He should have trusted her word rather than check on it himself 

Appealing to children's sense of fairness often helps in such situations 

Goals 

Definition : Purposes, intentions, or outcomes sought or desired; event or state desired; motive(s) of an 
agent in a situation; typically involves change in some aspect of the task environment from a cunent 
state to one desired by the adult. 

Content Focus : What purpose is accomfriished or is desired; the ends sought by actions; often stated in 
terms of need or want or the extent to which goals were or weren't achieved; consequences of actions 
on goals, on desired states. 

67 



ERLC 



77 



Appendix (ccntinued) 

Liyer 1. AdulNfooiscd pula: focused on adulf s needs, desires, states; on tasks of interest to 
the adult 

Illustrations: She wanted them to sit down and be orderly 
He was tiying to make things easier for himself 
She was trying to get this activity to work 
He wanted to get the job done 

Layer 2 . Child-focused goals in this situation: goals focused on children's feelings, 
perceptions, needs, development, interests, learning in this situation; context-bound 
child-focused goals. 

Illustrations: He wanted the girl in blue to feel good about her project 

She was concerned that Tommy would have a chance to explore it on his own 

She did it so they could learn each others' names 

He wanted them to work together 

His goal was for them to cooperate 

In onler for them to interact with each other 

Layer 3 . Child-focused goals not bound to a particular context or situation: goals focused on 

specific aspects of children's development and learning in general. 

Illustrations: These are good learning experiences for children to learn cooperation 

This helps children manap^e their environment 

This helps the children be self-sufficient 

This allows children to help each other 

Layer 4 . Goals stated in terms of broad, general areas of children's development: not bound to 
a particular context or situation. 

Illustrations: In order to enhance children's social development 

This contributed to the children's moral development 

Priorities 

Definition: Types of goal relationships; what goal is important to accomplish or what goal is mo*e 
important to accomplish in relation to other goals that could also be pursued. 
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Content Focui: What goals have priority over other goab, what outcomes are important; whoae needs 
have priority or importance; long- and short*term goals in tenns of ttietr relative importance; individual 
needs and group needs in terms of their relative importance; statements about ttie importance of 
developmental goals. 

Layer 1 . Statements that reflect priority on adult-focused needs, goals, nonchildren aspects of 
the task environment. 

Illustrations: The job needed to bt done; that took precedence 

Sanitation was sacrificed for letting ttie children do it on their o%vn 

Getting the children ready in time for ttieir parents was more important in this 
situation than developing their self-sufficiency 

Layer 2 . Child-focused statements expressing priority on short-term goals. contcxt-t)ound, 
situational goals, group needs; less developmentally significant goals or that detect priority on 
adult goals over child*focused goals. 

Illustrations: Meeting Cind/s need for comfort right then was the most important thing 

It was more important to her that the room be orderiy than that 
the children learn responsibility 

She was more concerned about her own needs than the children's 

Layer 3 . Child-focused statements expressing priority on lon^-term goals over short-term, 

context-bound, situational goals or more general short-term goals; individual needs over group 

needs; developmentally significant goals over less developmentally significant goals. 

Illustrations: While it caused them some distress at the moment, this contributed to 
children's learning to work out their own problems 

It is important for children to learn to enjoy an activity and to learn to do it on 
their own 

It is more important for one child to be able to express a trouUescMne thought, 
even when it takes awhile, and even if the group's play time is shortened that 
day 

This was an ideal opportunity for Christy to feel that she was an important 
part of the larger group even though it meant that she would miss her special 
time with her friend 

It was more important to support the children's social development in this 
situation than to worry about the noise 
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Uycrl Expression of priority on highly abstract, fteoretical-le vri child-focused goato. 
purposes, or intentions. 

Illustrations: She was really supporting ttie bo/s cognitive development but didn't seem to 
worry mudi about his social -'evdopment in the proccsi 

When possible, several areas of children's devdopment should be 
simultaneously addressed, rather than only one 

The chance to build trust among tfte children took precedence over supporting 
their independence 

Themes 

Definition : Generalized, overarching ideas, perspectives, views, assumptions and beliefs; reflect 
patterns over episodes that may vary greatly in context and specific aspects. 

Content Focus : Perspectives, assumptions, beliefs, views; attritnites characteristics, natures, or qualities; 
expectations of what people are like, and what, therefore, is likely to occur; opportunities and 
Gu/iculties often encountered in pursuing goals; imfdications, consequences, impact of goals on themes; 
understanding of sources of anothei^s goals; central concepts that summarize the meaning of an entire 
situation or that group a wide range of situations together. 

Layer 1 . Perspectives, assumptions, beliefs, views on children, adults or activities or the 
environment in this situation, on what they are, need, do, think, know, or can be, do, or think; 
attributes characteristics, natures, or roles to children or adults in this situation; context-bound 
in that reference as to what the person believes or assumes about children or adults in this 
situation. 

Illustrations: A lot of children didn't know it was clean up at all 
The children were self-motivated 
Some children did not want to clean up 

The children would have been able to do clean up without instructions from 
the teacher 

The children felt capable in their environment 

The children felt worthwhile, were relaxed 

The children were very capable of handling this activity 

The children needed encouraging 

The setting was geared to children 

Adult was low key, positive 
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Layer 2 - Perspectives, assumptions, beliefs, or views on what one can expect in gff^ff^ •bout 
childien or adults, their roles, what tfiey aie, need, do, think, know, or can be, dc, f>r tnink; 
what one can expect in general about situations or activities or environments involving 
cbUdim or adults; what opportunities and difficulties may arise; attributes characteristics or 
nahiies of children or adults generally; not context-bound; does not speak about children or 
adults in a particular sthiation but abotit children or adults in general; what the person believes 
or assumes about children or adults. 

Illustrations: I'd expect preschoolers to engage in a lot oi social play 

Qean up is one of those joba tfiat children are often not very excited about 

Children are valued, are capal>le 

Clean up can be a difficult transition 

All childim like to hear that they are good helpers 

Young people have feelings 

Young people should be recognized by name 

Children need prompting, to be told 

Layers . Perspectives about qualities of the relationships between adults and children, or the 
general climate surrounding interactions or sihiations that are reflected in a specific situation or 
that occur generally; general, overarching orientatioru of adults, not context-bound, but that 
are reflected acrjss situations; statement of the conditions governing the establishment of a role 
taken. 

Illustrations: A dimate of cooperation prevailed 

rhe situation was one in which children were controlled and order prevailed 

Guidance from a child's perspective entails sensitivity and responsiveness to 
children on the part of aduits 

Layer 4 , Large scale, ultimate consequences or implications of goals for children and of 
qualities of relationships; impact on children' ature, learning, development, their 
opportunities for development and learning; statements that focus on interests or other factors 
underiying qualities of relationships. 

Illustrations: The children come to rely on external direction rather than being self-directed 
The children get to relying on adult intervei.:;:.ii 
The children come to have a sense of control over what happens to them 
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